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1  Introduction 


This  report  documents  research  that  was  supported  by  the  Research  and  Plans  Division 
of  the  Program  Analysis  and  Evaluation  Directorate  of  the  U.S.  Army  Recruiting  Command 
(USAREC),  Fort  Knox,  KY  from  May  1993  through  30  September  1994.  The  basic  goal  of 
this  research  was  to  develop  a  decision-support  system  (essentially  a  model  plus  an  appropri¬ 
ate  database)  to  assist  in  the  selection  of  Regular  Army  (RA)  recruiters.  Ancillary  research 
on  this  subject  has  been  conducted  by  Professor  Glenn  F.  Lindsay  of  the  Operations  Re¬ 
search  Department  of  the  Naval  Postgraduate  School  (NPS)  and  is  reported  elsewhere  (see 
Lindsay  [1994]). 

The  research  approach  has  been  to  use  psychological-testing  methods  to  measure  the 
sales  ability  (as  a  critical  characteristic  necessary  for  recruiter  success)  of  recruiters  and 
assemble  a  database  to  support  the  development  of  a  multiple-linear-regression  model  to 
predict  recruiter  success  by  avoidance  of  failure.  The  Measure  Of  Recruiter  Effectiveness 
(MORE)  taken  for  this  work  has  been  the  probability  that  the  recruiter  is  successful  in 
meeting  an  assigned  quota  in  any  one  month  (see  Section  2.5  below).  Although  the  appro¬ 
priateness  of  this  MORE  has  changed  since  the  inception  of  the  work  reported  here,  the 
methodological  approach  developed  here  would  still  be  applicable  in  almost  all  foreseeable 
circumstances.  In  particular,  the  authors  believe  that  it  would  be  very  useful  for  USAREC’s 
“Recruit-the- Recruiter”  Program. 

Two  student  theses  have  supported  this  research,  namely 

1.  Todd  A.  Buchs  (OPT,  USA),  “Validation  and  Justification  of  the  Use  of  a 
Sales-Aptitude  Test  for  US  Army  Recruiter  Selection,”  M.S.  Thesis  in  Oper¬ 
ations  Research,  Naval  Postgraduate  School,  Monterey,  CA,  June  1994, 

and 

2.  Alejandro  S.  Hernandez  (CPT,  USA),  “Design  of  a  Predictive  Recruiter  Success 
Model  (PRiSM),”  M.S.  Thesis  in  Operations  Research,  Naval  Postgraduate  School, 
Monterey,  CA,  September  1994. 

The  interested  reader  can  find  virtually  complete  details  (e.g.  copy  of  sales-comprehension 
test,  most  sample  data,  at  least  examples  of  all  key  parts  of  the  recruiter  database  (including 
all  test-score  results),  virtually  all  statistical-analysis  results,  etc.)  in  these  theses.  Conse¬ 
quently,  it  would  seem  appropriate  to  have  the  report  at  hand  be  more  of  an  executive 
summary  in  content  and  style  of  presentation  than  of  a  detailed,  technical  report. 

The  primary  hmitation  of  the  work  at  hand  is  that  it  used  the  so-called  “current  em¬ 
ployee”  method  of  testing  U.S.  Army  recruiters  and  of  validating  the  model  subsequently 
developed.  Although  the  results  achieved  concerning  model  validity  were  extremely  positive, 
the  sample  of  recruiters  was  biased,  since  basically  only  surviving  successful  ones  were  tested. 
Consequently,  it  is  recommended  that  the  (so-called)  “follow-up”  method  (see  Section  2.6 
below)  of  sampling/validation  be  used  (whenever  feasible)  in  future  work,  since  USAREC 
does  not  currently  possess  appropriate  data  to  develop  an  entirely  valid  model  to  predict 
recruiter  success. 
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1.1  Background 


The  U.S.  Army  Recruiting  Command  (USAREC)  is  responsible  for  recruiting  and  pro¬ 
cessing  enlistees  into  the  U.S.  Army.  With  the  advent  of  the  all-volunteer  force,  its  mission 
has  attained  particular  significance,  since  volunteer  accessions  are  now  the  only  kind  of  per¬ 
sonnel  influx  into  the  Army.  With  the  end  of  the  Cold  War  and  the  downsizing  of  Department 
of  Defense  (DOD)  in  general  and  the  Army  in  particular,  it  has  become  imperative  that  the 
recruiting  process  be  an  efiicient  one. 

The  key  element  of  the  recruiting  process/system  is  the  individual  Army  recruiter  (typi¬ 
cally  a  sergeant  (i.e.  either  an  E5  or  an  E6)),  since  it  is  he  who  makes  personal  contact  with  a 
prospective  enlistee  and  “sells”  him/her  the  idea  of  an  Army  career.  Army  recruiters  recruit 
both  Regular  Army  (RA)  and  U.S.  Army  Reserve  (USAR)  enlisted  personnel.  However, 
only  RA  recruiters  are  used  to  recruit  RA  personnel,  and  only  USAR  recruiters  are  used  to 
recruit  USAR  personnel.  Army  (both  Regular  Army  (RA)  and  U.S.  Army  Reserve  (USAR)) 
recruiters  are  selected  (literally  “drafted”)  from  among  the  ranks  of  available  E5s  and  E6s, 
although  there  are  some  volunteers  (apparently  significantly  more  for  USAR  recruiters  than 
for  RA  recruiters).  Moreover,  according  to  USAREC  [1994],  the  current  recruiter-selection 
process  has  apparently  not  been  entirely  successful,  since  (1)  not  all  recruiter  selectees  suc¬ 
cessfully  complete  the  preliminary  Recruiter  School  and  (2)  not  all  those  who  do  successfully 
complete  Recruiter  School  become  successful  recruiters  (e.g.  consistently  meet  their  recruit¬ 
ing  quotas). 

In  either  case,  the  sergeant’s  Army  career  may  be  damaged  (frequently,  destroyed),  and 
the  Army  may  lose  (loses)  an  otherwise  very  effective  Non-Commissioned  Officer  (NCO)  (recruiters 
are  selected  from  the  top  ten  percent  of  personnel).  In  essence  then  the  process  of  selecting 
Army  recruiters  themselves  (as  well  as  that  of  managing  those  E5s  and  E6s  who  do  become 
recruiters)  is  an  inefficient  one.  Currently,  USAREC  does  not  have  any  methodology,  tool, 
or  model  to  aid  in  the  selection  of  recruiters  in  order  to  make  it  a  more  efficient  process. 

Moreover,  there  are  some  significant  differences  between  RA  and  USAR  recruiters.  RA 
recruiters  are  rotated  (serving  a  three-year  tour  of  duty)  in  a  given  U.S.  Army  Recruiting 
Station  (the  RA  recruiter  returning  to  his  previous  RA  career,  e.g.  to  serve  in  his  primary 
Military  Occupational  Specialty  (MOS)),  while  USAR  recruiters  are  permanently  assigned 
to  stations  located  near  their  permanent  civilian  residences.  More  importantly,  RA  recruiters 
are  drawn  from  a  population  of  career  soldiers  (i.e.  those  whose  primary  orientation  has  been 
to  make  a  career  out  of  the  Army),  while  USAR  recruiters  are  drawn  from  one  of  “week-end 
warriors”  (i.e.  those  whose  primary  orientation  has  been  towards  a  civilian  career,  with 
military  service  being  a  secondary  aspect).  Thus,  even  while  recruiting,  the  RA  recruiter 
undoubtedly  does  not  loose  sight  of  his  MOS  to  which  he  will  eventually  return.  Selling  is 
not  the  long-range  focal  point  of  his  life.  He  has  been,  is,  and  will  again  be  a  “warrior.” 
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1.2  Problem  Statement^* 


*  In  this  study  we  sought  to  determine  whether  data  currently  available  to  USAREC  will 
support  the  development  of  a  model  that  predicts  (1)  whether  a  particular  recruiter  selectee 
has  a  “good  chance”  of  completing  Recruiter  School  and  (2)  whether  a  particular  individual 
who  has  successfully  completed  it  has  a  “good  chance”  of  being  a  successful  recruiter.  Atten¬ 
tion  is  to  be  given  to  both  model  development  and  also  model  validation  in  this  evaluation  of 
currently  available  data.  System-effectiveness  measures  reflecting  overall  cost  to  the  Army 
will  be  investigated.  The  proposed  research  is  the  first  phase  (Phase  I)  of  a  three-phase 
program  to  develop  a  decision-support  system  to  aid  in  the  selection  of  Regular  Army  (RA) 
recruiters.  The  goal  of  Phase  I  is  to  develop  and  validate  the  above  model. 

1.3  Very  Brief  Review  of  Literature 

Testing  plays  a  prominent  role  in  several  of  the  Behavioral  Sciences,  including  Psychol¬ 
ogy  and  Personnel  Management,  in  which  aptitude  testing  is  important  (e.g.  see  Cronbach 
[1960],  Ghiselli  and  Brown  [1955],  Wigdor  and  Garner  [1982]).  Probably  the  largest  field  of 
application,  though,  is  higher  education  (see  Crawford  and  Burnham  [1973])  in  which  the 
Scholastic  Aptitude  Test  (SAT)  (see  Weitzman  [1982])  and  its  higher  derivatives  such  as  the 
Graduate  Record  Exam  (GRE)  have  been  given  to  tens  of  millions.  In  the  former  case  (i.e. 
the  SAT)  a  rather  loose  correlation  (multiple-correlation  coefficient  equal  to  approximately 
0.4  (Weitzman  [1982]))  has  been  found  between  math  and  verbal  SAT  scores  (independent 
variables)  and  success  in  college  (dependent  variable)  which  is  here  measured  by  the  first-year 
grade-point  average.  In  Personnel  Management  even  weaker  correlations  are  encountered: 
mean  correlation  coefficient  between  screening-test  result  and  performance  on  job  is  approxi¬ 
mately  equal  to  0.2  (Wigdor  and  Garner  [1982]).  Thus,  rather  weak  correlations  have  proven 
to  be  very  useful  in  major  fields  of  testing.  Such  weak  correlations  were  again  encountered 
in  the  work  reported  here. 

Moreover,  in  whatever  field  or  form  (psychological)  testing  takes  place,  it  is  invariably 
being  done  to  predict  future  performance  (for  example,  to  aid  in  the  selection  of  candidates 
to  fill  a  particular  position  [in  the  study  at  hand,  the  selection  of  U.S.  Army  recruiters]). 
Regression  models  (e.g.  see  Berry  and  Feldman  [1985]  and  also  Fox  [1991])  are  usually  used 
for  such  predictions,  and  hence  model  validity  is  a  particularly  important  issue  (as  it  is 
indeed  for  the  work  at  hand).  In  his  seminal  work  Essentials  of  Psychological  Testing,  L.J. 
Cronbach  [1960]  points  out  that  there  are  two  sampling  methods  for  validating  such  models 
(see  also  McKenna  [1967]):  (1)  the  current-employee  method  (also  frequently  referred  to  as 
the  concurrent  method  (e.g.  see  Atwater  et  al.  [1986]))  and  (2)  the  follow-up  method  (also 
frequently  referred  to  as  the  predictive  method). 

Furthermore,  the  U.S.  military  (i.e.  the  U.S.  Armed  Forces)  has  also  been  a  significant 
user  of  aptitude  testing,  with  every  serviceperson  (and  all  perspective  ones)  being  required  to 
take  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  (e.g.  see  USAREC  [1989,  pp. 
91-92])  (a  series  of  ten  aptitude  tests^).  The  ASVAB  rivals  the  SAT  in  the  number  of  people 

*To  avoid  distracting  the  reader,  all  footnotes  have  been  numbered  and  placed  together  just  before  the 
references. 
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it  is  given  to  every  year^.  Moreover,  such  aptitude  testing  for  job  placement  was  pioneered  by 
the  U.S.  Armed  Forces.  Today,  the  U.S.  Army  has  minimum-aptitude-score  requirements  for 
every  military  occupational  specialty  (MOS)  (see  USAREC  [1989,  p.  92])  except  recruiting'*. 
It  is  indeed  ironic  that  sales  aptitude  (one  of  the  main  ingredients  identified  by  USAREC 
Manual  100-5  as  being  necessary  for  successful  recruiting)  is  not  among  those  considered  by 
the  ASVAB^ 

A  widely  used  test  (that  is  relevant  to  the  work  at  hand)  in  the  commercial  world  is 
the  Sales  Comprehension  Test  developed  and  marketed  by  Martin  M.  Bruce  [1988]  (see  also 
Bruce  [1978]®).  It  is  by  far  the  most  widely  used  sales-comprehension  test^  and  undoubtedly 
the  most  convenient  to  use  (e.g.  see  Weitzman  [1994a]).  Most  test  subjects  only  require  15 
to  20  minutes  to  complete  the  test.  Further  information  (including  a  guide  to  the  literature 
of  this  test,  discussion  of  test  validity,  and  tables  of  norms  for  40  different  groups)  is  to  be 
found  in  the  “Examiner’s  Manual,  Sales  Comprehension  Test,  Revised”  (Bruce  [1978]). 

The  point  as  to  whether  similar  work  on  recruiter  selection  had  been  done  by  the  other 
U.S.  Services  was  also  investigated.  No  case  of  previous  sales-aptitude  testing  was  found.  A 
review  of  similar  work  done  at  NFS  appears  in  Hernandez  [1994].  U.S.  Navy  and  Marine 
Corps  work  on  development  of  a  recruiter-selection  battery  is  reported  in  Borman  et  al. 
[1979]  and  (a  different)  Borman  et  al.  [1981].  This  battery  of  tests  does  not  include  a  sales- 
aptitude  test,  but  rather  an  interest  inventory  (self-appraisal)  and  background  questionnaire. 
Recruiter  production  (the  dependent  variable)  was  measured  in  terms  of  average  number  of 
monthly  recruits  (i.e.  contracts  written)  and  a  subjective  appraisal  of  recruiter  performance 
(see  Borman  et  al.  [1976]).  Neither  of  these  measures  is  deemed  to  be  appropriate  for  the 
problem  at  hand  (see  Sections  1.2  and  also  1.4).  Favorable  results  for  validation  studies  (for 
both  concurrent  (i.e.  current-employee)  and  also  predictive  (i.e.  follow-up)  methods)  were 
reported  (Atwater  et  al.  [1986]). 

The  U.S.  Air  Force  has  done  no  sales-aptitude  testing  in  the  past  (primarily  because  it 
has  not  had  the  recruiter  problems  that  the  U.S.  Army  (e.g.  see  USAREC  [1994])  has  had) 
(Germadnik  [1994]),  but  plans  to  use  in  the  near  future  the  same  Sales  Comprehension  Test 
(i.e.  Bruce  [1988])  as  used  in  the  work  reported  here®.  Some  further  references  are  to  be 
found  in  Hernandez  [1994]  (see  also  Buchs  [1994]). 

1.4  Recruiter  Success 

For  convenience  and  simplicity,  the  term  “recruiter  success”  is  used  throughout  this 
report,  although  the  meaning  of  this  term  is  not  the  attainment  of  some  level  of  a  given 
“success  criterion,”  but  the  avoidance  of  a  some  level  of  a  given  “failure  criterion”:  the 
failure  to  “make  mission”  (i.e.  the  failure  to  meet  an  assigned  quota  of  recruits  in  a  given 
period  of  time).  Thus,  recruiter  success  is  used  in  the  sense  of  avoiding  a  given  failure 
criterion  for  a  given  period  of  time.  In  a  nutshell,  then,  “success”  actually  means  “avoidance 
of  failure.” 

Andy  Hernandez  [1994,  pp.  12-17]  has  investigated  the  details  of  “making  mission”  and 
their  impact  on  the  study  at  hand.  He  (Hernandez  [1994,  p.  13])  points  out  that  it  is  unclear 
what  the  precise  period  of  time  to  which  the  quota  refers  should  be.  In  fact,  a  recruiter  must 
make  seventeen  quotas  each  year;  twelve  monthly  quotas,  four  quarterly  quotas,  and  one 
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yearly  quota®.  Investigating  USAREC  regulations  and  manuals  and  Army  Regulation  (AR) 
601-1,  Hernandez  [1994,  p.  13]  found  that  USAREC  practice  considers  monthly  quotas  to 
be  the  critical  mission  for  the  recruiter.  Therefore,  this  study  took  recruiter  “success”  as 
referring  to  the  meeting  of  monthly  quotas  (each  and  every  month). 

The  above  criterion  for  recruiter  success  corresponds  to  the  situation  that  each  and  every 
USAREC  recruiter  has  an  individual  quota,  and  that  the  recruiter  is  evaluated  on  whether  or 
not  this  quota  is  met.  It  also  leads  to  the  two  measures  of  recruiter  effectiveness  (MOREs) 
discussed  in  Section  2.5  below.  With  the  advent  of  station  missioning  in  August  1994^°, 
however,  a  new  criterion  for  evaluating  recruiter  output  (and  also  a  new  MORE)  would  have 
to  be  developed  for  future  work. 

2  Research  Approach 


The  research  approach  used  for  the  problem  at  hand  (see  Section  1.2)  evolved  during 
the  conduct  of  the  work  reported  here.  The  initial  approach  was  to  try  to  use  existing  per¬ 
sonnel  records  (containing  mainly  biographical  data)  to  develop  a  multiple-linear-regression 
model  to  predict  recruiter  success  (see  Section  1.4).  The  final  approach  was  to  develop  a 
more  extensive  database  (including  results  of  a  sales-comprehension  test  and  also  Armed- 
Forces- Qualification-Test  (AFQT)  scores,  besides  the  usual  biographical  and  administrative 
information  contained  in  U.S.  Army  personnel  records.  The  use  of  the  Sales-Comprehension 
Test  (Bruce  [1988])  was  ground-breaking  for  recruiting  in  any  U.S.  Military  Service. 

The  evolution  from  our  initial  approach  (which  focused  on  existing  personnel  records)  to 
our  final  approach  (which  augmented  existing  personnel  records  with  various  aptitude-test 
results)  was  effected  by  investigating  two  basic  questions; 

(1)  Will  a  particular  candidate  recruiter  successfully  complete  Army 
Recruiter  School? 

(2)  Will  such  a  candidate  then  become  a  successful  recruiter? 

The  latter  question,  of  course,  depends  on  how  one  defines  “success”  (see  Section  1.4 
above).  Investigation  of  these  questions  led  us  to  the  fields  of  Personnel  Management  (e.g. 
see  GhiseUi  and  Brown  [1955])  and  Psychological  Testing  (e.g.  see  Cronbach  [I960]). 

Our  research  approach  evolved  in  the  following  manner.  The  original  question  asked  was, 
“Do  existing  personnel  records  contain  enough  information  to  predict  recruiter 
success?”  Investigation  of  this  latter  question  led  to  a  visit  to  the  Salinas  U.S.  Army  Re¬ 
cruiting  Station.  Informal  discussions  there  with  experienced,  successful  recruiting  sergeants 
then  revealed  that  they  thought  that  nothing  in  the  Enlisted  Database  would  be  use¬ 
ful  for  predicting  recruiter  success.  It  was  stressed  by  these  experienced,  successful 
recruiting  sergeants  that  sales  ability  was  an  important  factor  in  determining  whether  or  not 
a  recruiter  would  be  successful.  About  this  time  Professor  Bard  Mansager  obtained  some 
key  literature  from  USAREC  (e.g.  USAREC  [1989],  [1992])  and  suggested  that  all  members 
of  the  research  team  read  at  least  USAREC  Manual  100-5  (USAREC  [1989]). 
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Subsequently,  Professor  James  G.  Taylor  observed  that  candidate  recruiters  were  not 
screened  by  any  type  of  aptitude  testing  besides  that  required  to  qualify  for 
their  primary  MOS.  It  was  then  decided  that  we  should  focus  on  sales-aptitude  testing. 
Consequently,  Professor  Ron  Weitzman  [1994a]  investigated  which  sales-aptitude  test^^  was 
best  suited  to  the  project  at  hand.  The  Sales-Comprehension  Test  of  Martin  M.  Bruce 
[1988]  (see  also  Bruce  [1978])  was  selected  as  the  most  appropriate  test  instrument,  mainly 
because  of  the  facts  that  it  (1)  is  the  most  convenient  to  administer  (Weitzman  [1994a]),  (2) 
has  well-documented  validity  for  somewhat  similar  (civilian)  populations  of  sales  personnel 
(Bruce  [1978]),  and  (3)  is  difficult  to  “cheat”  on  (Weitzman  [1994a])  (see  also  Buchs  [1994, 

pp.  8-12]). 

Because  of  the  length  of  time  required  for  model/test  validation  by  the  “follow-up”  /predic¬ 
tive  method  (here  a  minimum  of  approximately  10  months),  it  was  decided  that  a  pilot  pro¬ 
gram  of  administering,  analyzing,  and  validating  (according  to  the  “current-employee”  / con¬ 
current  method)  the  commercially-available  sales-aptitude  test  should  be  pursued  in  the 
short  term.  Consequently,  Bruce’s  [1988]  Sales-Comprehension  Test  was  administered  by 
two  Master’s-Degree  thesis  students  in  the  Operations  Research  (OR)  Department  of  the 
Naval  Postgraduate  School  (NPS)  (Captains  Buchs  [1994]  and  Hernandez  [1994])  to  two 
USAREC  recruiting  battalions  and  a  control  group  (attendees  of  a  Basic  NCO  Course  (BN- 
COC)  at  Fort  Knox,  Kentucky  (i.e.  a  sample  drawn  from  the  same  pool  of  enlisted  soldiers 
from  which  recruiters  are  drawn)).  Visitation  to  brigade  headquarters  and  in-depth  dis¬ 
cussions  with  command  personnel  of  the  factors  influencing  recruiter  success  preceded  the 
testing.  These  efforts  were  enthusiastically  received  and  supported  in  the  field. 


2.1  Systems  Analysis 

This  section  briefly  highlights  a  systems  analysis  of  the  recruiter-selection-and-training 
process.  This  analysis  is  discussed  in  more  depth  in  Appendix  A  to  this  report.  The  main 
product  of  this  systems  analysis  is  a  graphical  depiction  (together  with  a  companion  narrative 
description)  of  the  recruiter-selection-and-training  process  (see  Figures  1  [Recruiter  Pre- 
Badge  Path]  and  2  [Recruiter  Post-Badge  Path]  of  Appendix  A).  In  other  words,  the  systems 
analysis  produces  a  (nonmathematical)  model  of  the  recruiter-selection-and-training  process 
(which  is  very  useful  for  subsequently  developing  any  mathematical  model,  discussing  any 
aspect  of  the  process,  etc.).  Some  recent  statistics  concerning  recruiter  “failures”  are  also 
given  in  Appendix  A  (see  Table  A-1),  with  recruiter-selection  criteria  being  given  in  Appendix 
B. 

The  graphical  depiction  allows  easy  visualization  of  the  sequence  and  flow  of  events 
in  the  recruiter-selection-and-training  process,  particularly  those  points  in  time  at  which 
individuals  leave  the  process  (e.g.  through  “failure,”  returning  to  troops  after  a  three- year 
tour  of  duty,  retirement  after  conversion  to  recruiting  MOS,  etc.).  It  has  been  the  key 
vehicle  for  developing  and  enhancing  our  understanding  of  this  important  process  and  has 
guided  aU  our  efforts  in  synchronizing  our  testing  efforts  with  the  process.  It  was  developed 
from  considering  various  USAREC  and  Department  of  the  Army  (DA)  publications  (e.g. 
USAREC  [1989],  [1992]),  augmented  by  conversations  with  field  personnel. 
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Finally,  the  graphical  depiction  of  the  recruiter-selection-and-training  process  is  impor¬ 
tant  for  planning  and  implementing  sales-aptitude  testing  according  to  the  follow-up  (or 
predictive)  method  (which  is  the  main  element  of  our  suggested  future  work  (see  Section  4)). 
Accordingly,  strategies  for  administering  the  Sales-Comprehension  Test  within  the  recruiter- 
selection-and-training  process  are  suggested  (again,  see  Appendix  A). 

2.2  Characteristics  of  a  Successful  Recruiter 

“USAREC  Manual  100-5  (Recruiting)”  (USAREC  [1989])  gives  the  following  four  char¬ 
acteristics  as  being  necessary  for  the  success  of  any  recruiter: 

(1)  salesmanship  ability, 

(2)  energy  and  enthusiasm, 

(3)  communications  skills, 

(4)  planning/organizing  skills. 

Unfortunately,  no  USAREC  publication  gives  any  guidance  for  operationalizing  (or  mea¬ 
suring)  the  extent  to  which  a  candidate  recruiter  possesses  any  of  these  essential  traits. 
Moreover,  it  is  not  even  clear  a  priori  that  the  above  are  the  only  significant  variables  af¬ 
fecting  recruiter  “success”  (see  Hernandez  [1994,  pp.  6-7  and  17-25]  for  further  details). 

Ideally,  one  would  want  some  type  of  test  instrument  to  measure  the  extent  to  which  a 
candidate  recruiter  possessed  each  of  the  above  traits  (attributes).  It  would  then  be  simply 
a  matter  of  determining  the  correlation  between  possession  of  these  traits  and  recruiter 
“success”  (e.g.  through  use  of  a  multiple-linear-regression  model)  in  order  to  able  to  predict 
recruiter  success.  Unfortunately,  such  test  instruments  do  not  currently  exist. 

For  convenience  and  because  numerous  USAREC  personnel  consistently  stressed  the 
importance  of  sales  ability  (combined  with  the  fact  that  our  research  had  revealed  that 
candidate-recruiter  sales  aptitude  had  not  been  measured  by  USAREC  previously),  the  re¬ 
search  at  hand  focused  on  the  measurement  of  sales  ability  with  the  Sales-Comprehension 
Test  developed  by  Martin  M.  Bruce  [1988]  (see  also  Bruce  [1978]).  For  the  development  of 
a  model  to  predict  recruiter  success,  other  measurable  variables  were  taken  as  surrogates  for 
the  other  characteristics  listed  above  (see  Hernandez  [1994,  pp.  6-7  and  17-25]  for  further 
details) . 


2.3  Sales- Aptitude  Test 

Sales  aptitude  was  measured  by  the  “Sales  Comprehension  Test,  Form  M,  Revised” 
(Bruce  [1988]),  which  is  commercially  available  through  Martin  M.  Bruce  of  Larchmont, 
New  York  (see  also  Bruce  [1978]).  This  test  was  chosen^^  because  it  is  not  very  demanding 
on  the  time  of  test  subjects  (requiring  about  15  to  20  minutes  for  completion),  is  easy  to 
administer  and  score,  inexpensive  to  purchase,  and  has  well-documented  validity,  reliability, 
and  norms  for  many  types  of  salespeople.  It  provides  an  objective  measure  (through  the 


9 


scoring  of  thirty  multiple-choice  questions)  of  the  subject’s  understanding  of  basic  principles 
of  selling  (Bruce  [1978,  p.  2]).  Although  it  is  possible  to  score  as  high  as  -1-93  or  as  low 
as  -145,  Bruce  [1978,  pp.  9-17)  uses  a  range  of  test  scores  from  -|-72  to  -26  for  depicting 
norms  (expressed  in  percentiles)  of  41  diflFerent  groups.  The  test  is  by  far  the  most  widely 
used  sales-comprehension  test^®  (Bruce  [1994])  and  has  been  used  by  many  hundreds  of  sales 
organizations,  with  an  extensive  literature  of  test  results  being  available  (see  Bruce  [1978] 
for  further  information). 

The  Sales-Comprehension  Test  has  been  designed^'^  for  salespeople  in  civilian  indus¬ 
try.  However,  Hernandez  [1994,  p.  4]  reports  that  discussions  with  Army- Recruiting-School 
(ARS)  instructors  and  recruiting-battalion  leaders  revealed  that  the  test  emphasizes  many 
of  the  same  skills  that  ARS  teaches  prospective  recruiters.  Moreover,  Bruce  [1978,  pp.  3-6] 
reports  that  the  test  has  been  validated  for  a  wide  spectrum  of  salespeople,  including  insur¬ 
ance  agents,  house-to-house  salesmen,  securities  salesmen,  and  copy-machine  salesmen.  Not 
all  validation  studies,  however,  reported  favorable  results.  Most  did,  though.  Discussing  the 
use  of  the  Sales-Comprehension  Test,  Bruce  says  that  maximal  value  will  be  obtained  when 
a  validity  study  precedes  operational  use  of  the  test.  He  also  points  out  that  it  measures  only 
one  aspect  (dimension)  of  sales  ability  (i.e.  sales  comprehension),  with  many  other  factors 
being  important  in  the  success  of  a  salesperson:  namely,  interest,  intelligence,  motivation, 
appearance,  health,  verbal  facility,  and  other  variables  (Bruce  [1978,  p.  2]).  Thus,  test 
validation  for  USAREC  recruiters  was  considered  to  be  an  essential  goal. 

Accordingly,  the  Sales-Comprehension  Test  was  administered  to  all  recruiters  in  two 
recruiting  battalions  (Baltimore  Battalion  of  First  Brigade  and  Santa  Anna  Battalion  of  Sixth 
Brigade)  and  also  to  a  control  group  representing  the  general  NCO  population  from  which 
recruiters  are  drawn  (all  attendees  of  a  Basic  Noncommissioned  Officer’s  Course  (BNCOC)  at 
Fort  Knox,  Kentucky,  in  February  1994).  Because  of  the  ground-breaking  nature  of  the  use 
of  such  a  test  within  USAREC,  extensive  coordination  (including  visitation  with  command 
personnel)  preceded  the  actual  administration  of  the  test.  The  test  itself  was  administered  on 
site  to  the  two  recruiting  battalions  by  two  U.S.  Army  Captains  (Todd  Buchs  [1994]  and  Andy 
Hernandez  [1994])  attending  the  Naval  Postgraduate  School  (Operations  Research /Systems 
Analysis  Curriculum).  USAREC  personnel  administered  the  test  (on  February  14,  1994)  on 
site  to  the  attendees  of  the  BNCOC  at  Fort  Knox. 

Thus,  Bruce’s  [1988]  Sales-Comprehension  Test  was  the  test  instrument  for  measuring 
sales  aptitude.  Moreover,  the  recruiter’s  score  on  the  Sales-Comprehension  Test 
was  the  primary  independent  variable  (also  called  predictive  variable)  for  the  research 
reported  here.  Further  details  concerning  the  Sales-Comprehension  Test  pertaining  to  our 
research  are  to  be  found  in  Buchs  [1994]  and  Hernandez  [1994]. 

2.4  Other  Independent  Variables 

Other  independent  variables  considered  for  the  development  of  the  multiple- linear-regression 
model  to  predict  recruiter  success  were  (Hernandez  [1994,  pp.  20-25]): 

(1)  Armed-Forces-Qualification-Test  (AFQT)  score. 
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(2)  length  of  time  in  service  before  becoming  recruiter  (in  months), 

(3)  Primary-Military-Occupational-Specialty  (PMOS)  variable, 

(4)  candidate-selection  variable, 

(5)  gender  variable. 

The  definitions  of  the  first  two  variables  are  straight-forward  and  require  no  further 
clarification.  The  last  three  variables  are  binary  variables,  only  taking  on  the  values  0  and 
1.  They  are  defined  as  follows 


{1  if  PMOS  is  for  a  combat  arm 

(1) 

0  if  PMOS  is  for  a  noncombat  arm 


r  1 


candidate  —  selection  variable  = 


0 


if  candidate  is  DA  selected 
if  candidate  has  volunteered 


(2) 


{1  if  candidate  is  male 

(3) 

0  if  candidate  is  female 

The  general  rationale  behind  the  selection  of  these  additional  independent 
variables  was  that  together  with  the  Sales-Comprehension-Test  score  they  mea¬ 
sure  the  extent  of  possession  of  all  the  characteristics  considered  to  be  essential 
for  a  recruiter  to  be  successful  (see  Section  2.2  above).  The  specific  rationales  behind 
the  selection  of  the  above  five  additional  independent  variables  is  as  follows.  The  recruiter’s 
Armed-Forces-Qualification-Test  (AFQT)  score  was  taken  as  a  surrogate  for  both  communi¬ 
cations  skills  and  also  planning/organizing  skills  (see  Section  2.2).  The  AFQT  is  a  subset  of 
the  of  the  ASVAB  (see  Section  1.3  above),  measuring  intellectual  capability  in  terms  of  ver¬ 
bal  and  quantitative  abilities^^.  Verbal  ability  is  the  cornerstone  of  effective  communication, 
which  is  essential  for  teaching,  persuading,  enticing,  etc.  potential  enlistees.  Additionally, 
intellectual  ability  (i.e.  both  verbal  and  quantitative  abilities)  is  essential  for  coping  with 
the  complexities  of  planning  and  organizing.  Moreover,  Eitelberg  [1988,  p.  30]  reports  that 
there  is  a  high  correlation  between  AFQT  scores  and  trainability  (i.e.  soldiers  who  score  well 
on  the  AFQT  usually  achieve  higher  scores  in  service  schools). 

The  soldier’s  length  of  service  before  becoming  a  recruiter  (in  months)  was  taken  as  a 
surrogate  for  planning/organizing  skills,  since  the  longer  a  soldier  is  in  the  service,  the  greater 
the  demands  for  having  developed  such  skills  (Hernandez  [1994,  p.  19]).  Finally,  the  last 
three  independent  variables  listed  above  (all  binary  variables)  were  considered  to  measure 
intangibles.  C.  Spearman  [1904]  calls  such  intangibles  “non-intellective  factors”  and  has 
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developed  a  theory  of  general  intelligence  (Spearman  [1927])  that  suggests  many  interesting 
ideas  on  how  non-intellective  factors  (e.g.  motivation,  interest,  energy,  enthusiasm,  etc.)  can 
determine  human  performance. 

Moreover,  an  initial  screening  of  the  potential  significance  of  these  last  three  independent 
variables  was  carried  out  before  the  development  of  the  multiple-linear-regression  model. 
This  screening  consisted  of  a  so-called  expectancy-chart  analysis  (e.g.  see  McKenna  [1967, 
p.  61])  and  was  carried  out  using  recruiter  output  (i.e.  the  dependent  variable)  in  terms  of 
the  probability  of  making  the  monthly  mission  and  also  the  overall  average  probability  of 
success  for  all  recruiters  in  the  database  of  0.6848.  This  analysis  (done  for  each  of  the  three 
variables  classified  as  a  non-intellective  factor)  supported  their  consideration  as  predictive 
variables.  See  Hernandez  [1994,  pp.  12-25  and  Appendix  G  (also  Appendix  F)]  for  further 
details. 

Thus,  the  independent  variables  discussed  in  these  last  two  sections  (i.e.  Sections  2.3  and 
2.4)  provided  quantitative  measures  of  the  extent  to  which  a  (candidate)  recruiter  possesses 
the  four  characteristics  that  USAREC  experience  has  found  necessary  for  success  (see  Section 
2.2  above). 

2.5  Measures  of  Recruiter  Effectiveness  (MQREs) 

The  dependent  variable  (reflecting  the  output  of  a  recruiter)  used  in  the  work  at  hand 
was  the  probability  of  making  mission  (i.e.  meeting  the  monthly  quota  (see  Section  1.4 
above)).  In  a  nutshell,  this  is  the  probabihty  that  the  recruiter  is  “successful”  in  any  given 
month.  In  the  field  of  personnel  management  (e.g.  see  McKenna  [1967])  such  a  quantity  is 
referred  to  as  a  “performance  criterion.”  However,  the  field  of  operations  research^®  refers  to 
the  quantitative  expression  of  the  extent  to  which  specific  mission  requirements 
have  been  attained  by  a  system  as  a  “measure  of  effectiveness  (MOE)”  (Taylor 
[1973],  [1992]).  The  essential  difference  between  performance  and  effectiveness  is  that  the 
latter  gives  consideration  to  system  goals^^. 

Accordingly,  we  will  refer  to  the  dependent  variable  (here  the  probability  of  a 
recruiter  making  his/her  monthly  quota^®)  as  estimating  a  Measure  Of  Recruiter  Ef¬ 
fectiveness  (MORE),  since  it  measures  the  extent  of  a  recruiter’s  mission  (i.e.  goal) 
accomplishment.  This  MORE  was  estimated  by  the  ratio  of  the  total  number  of  times  that  a 
recruiter’s  monthly  mission  was  accomplished  divided  by  the  total  number  of  months  under 
consideration. 

However,  a  recruiter’s  mission  is  actually  composed  to  two  major  categories  of  enlistees: 
(1)  Graduate  or  Senior  Categories  I-IIIA  (GSA)  and  (2)  all  other  categories  of  enlistees.  The 
first  category  (GSA)  consists  of  those  individuals  who  have  scored  above  the  50*^  percentile 
on  the  AFQT  and  who  are  either  current  high-school  seniors  (with  a  specified  graduation 
date)  or  at  least  the  equivalent  of  a  high-school  graduate.  The  second  consists  of  all  other 
categories  of  enlistees  and  will  be  referred  to  here  as  the  Volume  (V OL)  Category  (see  Buchs 
[1994,  pp.  22-23],  Hernandez  [1994,  p.  15]  for  further  details). 

Thus,  since  a  recruiter’s  mission  is  composed  of  these  two  categories  of  enlistees  (namely, 
GSA  and  VOL),  a  composite  (referred  to  here  as  “combined”)  probability  of  making  mission 
had  to  be  considered. 
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This  combined  probability  was  computed  using  current  Department  of  the  Army  policy 
fractions  of  0.67  and  0.33  corresponding  to  the  GSA  and  VOL  categories,  respectively.  Hence 
(see  Buchs  [1994,  p.  27],  Hernandez  [1994,  p.  15]) 

(combined  probability  of  making  mission)  = 


(0.67)  X  Prob  [recruiter  makes  GSA  monthly  mission]  + 
(0.33)  X  Prob  [recruiter  makes  VOL  monthly  mission] 


(4) 


Thus,  the  actual  MORE  (or  dependent  variable)  used  in  the  work  at  hand  was  the 
combined  probability  of  making  mission  given  by  equation  (4)  above. 


2,6  Sampling/ Validation  Methods 

There  are  two  methods  for  validating  models  that  predict  human  performance  (e.g.  see 
Cronbach  [1960]  and  McKenna  [1967]) 

(1)  current-employee  method  (also  referred  to  as  concurrent  method^®). 


and 

(2)  follow-up  method  (also  referred  to  as  predictive  method^®). 

The  first  method  (i.e.  current-employee  method)  tests  (samples  from)  current  employees 
and  uses  past  performance  data  on  them  for  model  validation.  It  has  the  advantage  of  having 
minimal  resource  (i.e.  time  and  effort)  requirements  for  data  collection,  since  existing  records 
can  be  used.  Unfortunately,  it  has  the  disadvantage  for  the  work  at  hand  of  not  including 
any  unsuccessful  recruiters  in  the  database. 

The  second  method  (i.e.  the  follow-up  method)  tests  (samples  from)  selectees  and  then. 
while  they  are  at  school  or  at  work,  starts  to  collect  performance  data  for  model  validation. 
Thus,  the  “follow-up”  method  uses  future  performance  data,  and  one  must  wait  for  this 
data  to  be  collected.  If  one  were  to  use  this  method  and  tested  candidate  recruiters  just 
before  entry  into  the  Army  Recruiting  Course  (ARC),  then  one  would  have  to  wait  for  at 
least  11  months  to  find  out  which  candidate  recruiters  failed  either  the  ARC  or  during  the 
nine-month  TTE  period  (see  Appendix  A).  See  Buchs  [1994,  pp.  12-14]  for  a  more  detailed 
discussion  of  the  pros  and  cons  of  each  of  these  two  methods. 

Additionally,  judgment  sampling^^  was  used  to  select  the  two  battalions  and  also  the 
control  group  to  which  the  Sales-Aptitude  Test  was  administered.  Specifically,  data  was 
collected  from  two  recruiting  battalions^^  (i.e.  the  Sales-Comprehension  Test  was  adminis¬ 
tered  to  all  recruiters  in  them),  which  were  judgmentally  selected  from  among  USAREC’s 
approximately  (at  that  time)  40  recruiting  battalions.  These  two  battalions  were  selected 
to  represent  an  “outstanding”  and  also  an  “average”  recruiting  battalion.  Thus,  these  two 
battalions  were  a  judgment  sample^®  from  USAREC’s  recruiting  battalions.  Addition¬ 
ally,  data  was  also  collected  from  a  control  group^'^  representing  the  general  population  of 
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sergeants  from  which  recruiters  are  drawn  (i.e.  the  E5/E6  pool  of  Appendix  A).  This  control 
group  was  also  a  judgment  sample. 

Circumstances  (i.e.  lack  of  sufficient  time)  dictated  that  the  current-employee  method  be 
used  for  the  study  at  hand.  Consequently,  no  data  was  collected  for  unsuccessful  recruiters, 
and  this  situation  may  have  diminished  results.  However,  our  work  here  does  show  the  great 
potential  of  personnel  testing  for  helping  to  solve  USAREC  problems.  It  is  most  strongly 
recommended  that  the  follow-up  method  be  used  whenever  possible  in  future 
work. 


2.7  Data  Collection  and  Database  Development 

Field  data  collection  consisted  of  administering  Bruce’s  [1988]  Sales- Comprehension  Test 
to  the  two  selected  recruiting  battalions  by  U.S.  Army  Captains  Buchs  and  Hernandez 
and  also  to  the  BNCOC  attendees  by  USAREC  personnel.  More  specifically,  the  Sales- 
Comprehension  Test  was  given  to  131  recruiters  in  the  Baltimore  Battalion  of  the  First 
Recruiting  Brigade  and  to  145  recruiters  in  the  Santa  Ana  Battalion  of  the  Sixth  Recruiting 
Brigade.  Additionally,  it  was  given  to  54  sergeants  in  the  BNCOC  at  Fort  Knox,  Ken¬ 
tucky,  of  which  31  did  not  have  any  prior  sales  experience  and  23  did  (including  6  former 
recruiters).  Around  this  basic  sales-aptitude  data  were  constructed  two  databases:  Buchs’s 
[1994]  database  and  Hernandez’s  [1994]  database. 

The  data  in  Buchs’s  database  consisted  of  Sales-Comprehension-Test  scores  (see  Buchs 
[1994,  Appendix  C]),  data  from  a  “respondent  data  form”  given  to  each  person  who  took 
the  Sales-Comprehension  Test  (see  Buchs  [1994,  Appendix  D])  and  administrative  and  per¬ 
sonnel  data  provided  by  the  battalion  tested  and  also  the  Program- Analysis-and-Evaluation 
(PA&E)  Directorate  of  Headquarters  USAREC.  There  were  two  major  subdivisions  in  Buchs’s 
database:  one  part  for  data  on  recruiters  and  the  other  for  nonrecruiters  (i.e.  attendees  of 
the  BNCOC).  The  recruiter  database  contained  data  on  14  variables  for  each  recruiter  (see 
Buchs  [1994,  pp.  17-19]  for  further  details.  It  should  be  noted  that  Buchs’s  MORE  2  is 
the  Measure  of  Recruiter  Effectiveness  (MORE)  given  by  equation  (4)  in  Section  2.5  above 
(i.e.  the  “combined”  probability  of  making  the  monthly  mission).  The  nonrecruiter  database 
contained  data  on  three  variables  (see  Buchs  [1994,  p.  20]). 

Hernandez’s  database  contained  all  Buchs’s  recruiter  data  plus  additional  administra¬ 
tive  data  from  the  tested  battalions  and  also  ASVAB  and  AFQT  scores  obtained  from  the 
Defense  Manpower  Data  Center  (DMDC)  and  also  the  Army  Research  Institute  (ARI).  Her¬ 
nandez’s  thesis  contains  his  “initial  database  for  276  recruiters”  (Hernandez  [1994,  Appendix 
A])  (which  essentially  amounts  to  Buchs’s  database  for  recruiters),  as  well  as  his  “reduced 
database  for  101  recruiters”  (Hernandez  [1994,  Appendix  D])  (which  consisted  of  data  on  the 
six  independent  variables  discussed  in  Sections  2.3  and  2.4  above  and  also  the  corresponding 
MORE  for  the  101  recruiters  for  whom  complete  TTE-period^^  records  were  available). 
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3  Results  of  Data  Analysis 


Data-analysis  results  are  highlighted  in  this  section,  with  complete  details  (including 
samples  of  and  sometimes  large  parts  of  the  databases  involved^®)  being  available  in  the 
two  student  theses  cited  in  Section  1  above;  Buchs  [1994]  and  Hernandez  [1994].  The  first 
of  these  theses  (namely,  Buchs  [1994])  investigated  whether  the  Sales-Comprehension  Test 
(SCT)  by  itself  could  predict  recruiter  success.  The  current-employee/concurrent  sampling 
method  was  used  for  both  these  theses.  Thus,  only  (currently)  successful  recruiters  were 
available  for  testing^^.  Buchs’s  thesis  also  contains  a  preliminary  statistical  screening  of  the 
Sales-Comprehension-Test-score  data,  which  provides  some  valuable  insights  into  the  general 
nature  of  sales  comprehension  and  ability  (and  their  measurement)  and  the  possession  of 
these  essential  capabilities  by  USAREC  personnel. 

The  second  thesis  (Hernandez  [1994])  investigated  whether  a  multiple-linear-regression 
model  could  be  developed  for  a  database  that  included  results  of  the  Sales-Comprehension 
Test  and  also  Armed-Forces-Qualification-Test  (AFQT)  scores,  besides  the  usual  biographical 
and  administrative  information  contained  in  U.S.  Army  personnel  records  (see  Section  2.7 
above).  The  same  sales-comprehension-test  scores  as  used  by  Buchs  were  used  by  Hernandez. 

More  precisely,  Hernandez  used  a  subset  of  Buchs’s  SCT-score  data  (that  was  considered 
to  be  most  germane  to  investigating  the  validity  of  his  multiple-linear-regression  model), 
augmented  by  some  additional  (readily-obtainable)  personnel  data  on  the  recruiters  tested. 
One  important  contribution  of  Hernandez’s  work  was  the  creation  of  a  basic  database  (in¬ 
cluding  both  SCT  and  also  AFQT  scores)  relating  recruiter  characteristics  and  capabilities 
to  performance.  No  such  database  had  heretofore  existed.  Because  of  the  data  limitations 
noted  above  (i.e.  use  of  the  current-employee  method  of  testing),  complete  statistical  details 
will  not  be  given  here  (even  for  the  highlighted  results),  since  data  quality  may  not  warrant 
it. 


3.1  Recruiter  Sales  Comprehension  and  General  Test  Validity 

Knowledge  of  recruiter  sales  comprehension  and  ability  is  critical  for  developing  and 
using  any  model  to  predict  recruiter  success.  Do  some  recruiting  battalions  just  have  (or 
attract)  recruiters  with  greater  recruiting  abilities?  Moreover,  are  Sales- Comprehension- 
Test  (SCT)  scores  stable  over  time  (suggesting  that  sales  comprehension  (like  intelligence) 
is  innate  to  a  recruiter)?  Or,  can  they  change  (suggesting  that  sales  comprehension  can  be 
acquired/learned)?  Can  the  Sales-Comprehension  Test  (SCT)  distinguish  between  people 
with  sales  experience  and  those  without  it?  Such  important  questions  were  investigated  by 
a  general  analysis  of  the  test  data^. 

Basic  descriptive  statistics  (essentially  means  and  standard  deviations)  were  computed 
for  the  SCT  data  collected  from  each  of  the  two  recruiting  battalions  tested  (and  also  for 
the  output,  or  dependent,  variable  discussed  in  Section  2.5  above)  (Buchs  [1994,  p.  56]). 
Results  are  shown  in  Table  I  below.  For  our  study  reported  here,  the  combined  probability 
of  making  mission  was  taken  as  the  basic  measure  of  recruiter  effectiveness^®  (MORE)  and 
(for  a  given  recruiter)  is  given  by  equation  (4)  above. 
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TABLE  I:  Means  and  Standard  Deviations  for  the  Two  Battalions  that  were  given 
Sales-Comprehension  Test 


Test  Score 

Combined  Probabilty 

of  Making  Mission 

Battalion 

Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 

Baltimore 
(n  =  131) 

21.27 

19.91 

0.738 

0.136 

Santa  Ana 
(n  =  145) 

20.10 

18.00 

0.681 

0.142 

Statistical  analysis  (Buchs  [1994,  pp.  36-37  (see  also  p.  94)])  (using^°  a  =  0.05)  revealed 
that  there  is  no  statistically-significant  difference  in  the  means  and  standard  deviations  for 
test  scores  of  the  two  battalions  (i.e.  all  test-score  data  is  essentially  drawn  from  similar 
sources  (i.e.  population)). 

Consequently,  test  data  for  the  two  battalions  was  combined  for  subsequent  analyses 
(by  both  Buchs  [1994]  and  also  Hernandez  [1994]),  especially  when  sample  sizes  in  some 
cases  became  perilously  small.  Thus,  on  the  basis  of  statistical  analysis  of  test  data  for 
two  USAREC  battalions,  we  have  tentatively  concluded  that  sales  comprehension  of 
recruiters  is  possibly  the  same  in  all  USAREC  recruiting  battalions.  For  many 
USAREC  purposes,  such  a  result  can  be  extremely  valuable. 

Do  non-recruiters  have  the  same  level  of  sales  comprehension  as  recruiters  do?  One  would 
expect  that  (by  the  very  nature  of  the  different  mental  and  emotional  environments  of  the 
soldiers  in  question)  they  do  not  have  the  same  level  of  sales  comprehension  and  consequently 
will  score  (statistically)  differently  on  the  Sales-Comprehension  Test.  Moreover,  Bruce  [1978, 
pp.  3-4]  has  reported  that  his  Sales-Comprehension  Test  will  distinguish  between  sales  and 
non-sales  personnel  (which  is  taken  as  an  indication  of  the  test’s  status  validity  (American 
Psychological  Association  [1966])). 

Hence,  the  Sales-Comprehension  Test  was  also  administered  to  a  control  group  represent¬ 
ing  the  general  NCO  population  from  which  recruiters  are  drawn  (all  attendees  of  a  Basic 
Noncommissioned  Officer’s  Course  (BNCOC)  at  Fort  Knox,  Kentucky,  in  February  1994,  and 
all  these  attendees  currently  in  non-sales  jobs/MOSs).  Some  BNCOC  attendees,  though,  did 
have  prior  sales  experience  or  had  even  been  recruiters  (see  Buchs  [1994,  especially  pp.  84 
and  87]  for  further  details).  Some  basic  descriptive  results  (from  Buchs  [1994,  p.  58])  are 
shown  in  Table  II  below.  Statistical  analysis®^  has  revealed  that  the  difference  between  the 
mean  test  scores  of  recruiters  and  non-recruiters  is  highly  statistically  significant.  It  is  also 
test-scorewise  quite  substantial. 
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Table  II.  Some  Basic  Descriptive  Statistics  for  Sales-Comprehension-Test 
Scores  of  Recruiters  and  Non-Recruiters 


Sample 

Group 

Test  Score 

Mean 

Standard 

Deviation 

Maximum 

Value 

Minimum 

Value 

Non-Recruiters 
(n  =  54) 

-3.15 

26.41 

47 

-90 

Recruiters 
(n  =  276) 

20.66 

18.93 

64 

-38 

Thus,  one  can  safely  conclude  (at  least  tentatively)  that  recruiters  and  non-recruiters 
score  substantially  differently  on  Bruce’s  [1988]  Sales- Comprehension  Test.  As  noted  above, 
this  fact  can  then  be  taken  as  an  indication  of  the  test’s  status  validity  (American  Psycho¬ 
logical  Association  [1966]).  Hence,  (in  some  sense)  the  general  validity  of  the  Sales- 
Comprehension  Test  has  been  established.  It  remains,  though,  to  investigate  whether 
or  not  the  test  (by  itself  or  in  combination  with  other  indicators)  has  enough  predictive 
capability  for  USAREC  purposes  (i.e.  whether  the  test  can  be  used  to  help  predict  recruiter 
success). 

First,  however,  it  will  be  instructive  to  consider  the  various  categories  of  recruiters 
(and  also  non-recruiters)  considered  by  Buchs.  In  his  investigation  of  whether  the  Sales- 
Comprehension  Test  by  itself  could  predict  recruiter  success,  Buchs  [1994,  p.  29]  found 
it  natural  (and  most  appropriate)  to  group  his  data  on  recruiters  according  to  whether  a 
recruiter  was  currently  in 

(1)  the  TTE  period, 

(2)  his/her  first  recruiting  tour  of  duty  (but  after  the  TTE  period),  or 

(3)  a  recruiting  tour  of  duty  beyond  the  first  (and  is  a  career  recruiter). 

Because  of  the  computational  requirements  for  one  of  the  Measures  Of  Recruiter  Effec¬ 
tiveness  (MOREs)  (combined  cumulative  percent  success  (see  Buchs  [1994,  pp.  24-261])) 
used  in  his  work,  this  classification  of  recruiters  (and  hence  the  test  data)  seemed  natural  to 
Buchs.  Expressed  another  way,  recruiters  could  equivalently  be  classified  as 

(1)  recruiters  with  0  months  to  9  months  recruiting  experience, 

(2)  recruiters  with  10  months  to  24  months  recruiting  experience,  or 

(3)  recruiters  with  greater  than  24  months  recruiting  experience. 

For  brevity,  these  recruiter  categories  were  usually  referred  to  as 

(1)  recruiters  (0-9  months). 
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(2)  recruiters  (10-24  months),  and 

(3)  recruiters  (>  24  months). 


Data  was  also  available  (e.g.  see  Buchs  [1994,  p.  20])  to  consider  various  categories 
of  non-recruiters  (i.e.  BNCOC  attendees  who  were  tested).  Moreover,  it  is  instructive  to 
consider  (based  on  the  above  classification  considerations)  test-score  descriptive  results  for 
these  various  categories  of  recruiters  and  non-recruiters.  Some  basic  descriptive  results  are 
consequently  shown  in  Table  III  below. 


Table  III.  Some  Basic  Descriptive  Statistics  for  Sales-Comprehension-Test 
Scores  for  Various  Categories  of  Recruiters  and  Non- Recruiters 


Sample 

Group 

Test  Score 

Mean 

Standard 

Deviation 

Maximum 

Value 

Minimum 

Value 

Non-Recruiters  (All) 

-3.15 

47 

-90 

Non-Recruiters  with  no 
Sales  Experience 

-12.39 

27 

-90 

Non-Recruiters  with 
Sales  Experience 

9.30 

20.77 

47 

-24 

Non-Recruiters  with 
Sales  Experience 
(Less  Past  Recruiters) 

12.56 

19.57 

47 

-16 

Recruiters  (0-9  months) 

20.59 

22.54 

63 

-32 

Recruiters  (10-24  months) 

20.48 

18.75 

64 

-38 

Recruiters  (>  24  months) 

20.82 

18.35 

63 

-26 

Focusing  on  the  above  three  categories  of  recruiters,  the  same  basic  descriptive  results 
are  shown  in  Table  IV  below,  with  less  clutter  from  non-recruiter  results.  Statistical  analysis 


Table  IV.  Some  Basic  Descriptive  Statistics  for  Sales-Comprehension-Test  Scores 
for  the  Three  Categories  of  Recruiters 


Sample 

Group 

Test  Score 

Mean 

Standard 

Deviation 

Maximum 

Value 

Minimum 

Value 

Recruiters  (0-9  months) 

(n  =  28) 

20.59 

22.54 

63 

-32 

Recruiters  (10-24  months) 
(n  =  116) 

20.48 

18.75 

64 

-38 

Recruiters  (>  24  months) 
(n  =  132) 

20.82 

18.35 

63 

-26 
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(Buchs  [1994,  pp. 51-52])  revealed  that  these  three  length-of-recruiting-experience-based  cat¬ 
egories  of  recruiters  have  basically  the  same  mean  test  scores  (and  also  standard  deviations). 
Thus,  as  far  as  test  scores  are  concerned,  recruiters  (even  with  quite  different  amounts  of 
recruiting  experience)  are  essentially  a  very  homogeneous  group.  This  result  also  suggests 
(but  there  is  no  temporal  data  to  confirm  the  hypothesis)  that  a  recruiter’s  test  score  is  con¬ 
stant  over  time.  One  would  have  to  do  some  type  of  follow-up®^  testing  to  firmly  establish 
such  a  fact  (i.e.  the  constancy  of  a  recruiter’s  Sales-Comprehension-Test  score  over  time). 

Such  insights  (i.e.  how  test  scores  depend  (or  do  not  depend)  on  amount  of  recruiting 
experience)  can  be  extremely  valuable  for  model  (to  predict  recruiter  success)  development 
and  validation  (as  well  as  for  managing  recruiters  in  general).  We  have  barely  scratched  the 
surface  here  and  much  more  important  work  remains  to  be  done. 

3.2  Prediction  of  Recruiter  Success  by  Test  Score  Alone 

Initially  we  had  thought  that  recruiter  success  would  be  highly  correlated  with  Sales- 
Comprehension-Test  score  (henceforth  simply  denoted  as  “test  score”  when  no  ambiguity 
exists).  Thus,  our  initial  working  hypothesis  was  that  there  would  be  a  positive  correlation 
between  test  score  and  recruiter  performance  as  measured  by  our  Measure  Of  Recruiter 
Effectiveness  (MORE)  given  by  equation  (4).  Later  we  found  that  Martin  Bruce  [1978,  p. 
2]  (creator  of  by  far  the  most  widely-used  sales-comprehension  test  (and  consequently  an 
authority  on  predicting  the  success  of  sales  personnel))  had  specifically  emphasized  that 
many  other  factors  besides  sales  comprehension  are  important  for  success  as  a  sales-person: 
interest,  intelligence,  motivation,  appearance,  health,  verbal  facility,  etc.  The  thesis  results 
of  Buchs  [1994]  and  Hernandez  [1994]  have  corroborated  Bruce’s  statement. 

After  collecting  data  (both  test  scores  and  also  performance  data  (in  the  latter  case, 
values  for  our  MORE),  scatter  plots  (see  Buchs  [1994,  pp.  117, 123,  and  129])  of  test  score 
versus  MORE-value  (i.e.  the  composite  probability  that  a  particular  individual 
makes  mission  (as  given  by  equation  (4)))  were  constructed.  These  scatter  plots,  however, 
displayed  little  trend  (i.e.  relationship)  between  the  values  of  these  two  variables. 

Moreover,  an  investigation  of  correlation  coefficients  (see  Table  V  below)  statistically 
confirmed  the  obvious  (as  shown  in  the  scatter  plots  discussed  above):  a  recruiter’s  test 
score  had  little  correlation  with  his/her  performance  as  measured  by  our  MORE. 

The  P-values  quoted  in  Table  V  above  correspond  to  the  hypothesis  test  (two-tailed) 
that  the  population  correlation  coefficient  is  equal  to  zero  (see  Buchs  [1994,  pp.  31-32]  for 
further  details).  Spearman’s  rank  correlation  coefficient  was  used  to  estimate  this  population 
parameter  if  the  data  suggested  departure  from  normality  in  the  underlying  population 
(statistical  testing  involved  here  too).  Otherwise,  Pearson’s  correlation  coefficient  (for  normal 
populations)  was  used  (again,  see  Buchs  [1994,  pp.  32-34]  for  further  details).  The  P-value 
results  given  in  Table  V  indicate  that  one  cannot  reject  the  hypothesis  (at  a  significance 
level,  a  =  0.05)  that  there  is  no  (i.e.  zero)  correlation  between  a  recruiter’s  test  score  and 
performance  measured  by  our  MORE. 
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Table  V.  Correlation  between  Sales-Comprehension-Test  Score  and  Combined 
Probability  of  Making  Mission  (MORE  Given  by  Equation(4)) 


Sample  Group 

Pearson’s 

Correlation 

Coefficient 

2-Tailed 

P- Value  for 
P’s  C.  C. 

Spearmans’s 

Correlation 

Coefficient 

2-Tailed 
P-Value  for 
S’s  C.  C. 

Recruiters  (0-9  months) 

(n  =  103) 

— 

— 

0.11 

0.26 

Recruiters  (10-24  months) 
(n  =  113) 

-0.01 

0.91 

— 

— 

Recruiters  (>  24  months) 
(n  =  131) 

-0.06 

0.47 

One  point  concerning  the  sample  size  for  the  sample  group  “Recruiters  (0-9  months)” 
(i.e.  recruiters  currently  in  the  TTE  period)  merits  further  discussion  here.  Although  only 
28  out  of  the  276  recruiters  tested  were  actually  currently  in  their  TTE  period,  USAREC 
records  did  provide  TTE-period  performance  data  (and  hence  a  value  for  the  MORE  could 
be  estimated)  for  many  field  recruiters  (i.e.  recruiters  who  had  already  completed  their 
TTE  period).  Since  the  data  suggested  that  a  recruiter’s  Sales-Comprehension-Test  score  is 
constant  over  time  (see  discussion  in  Section  3.1  above,  just  after  Table  IV),  past  TTE-period 
performance  data  was  consequently  combined  with  test  scores  for  those  recruiters  for  whom 
the  appropriate  TTE-period  records  existed  (see  Buchs  [1994,  p.  39]  for  further  discussion). 
These  considerations  led  to  a  larger  sample  size  for  the  recruiter  category  ’’Recruiters  (0-9 
months”  than  the  number  of  recruiters  (who  were  currently  in  their  TTE  period)  actually 
tested. 

Finally,  Buchs  [1994,  pp.  54-55]  showed  that  the  Sales-Comprehension  Test  could  not 
distinguish  between  successful  and  unsuccessful  recruiters  (success/nonsuccess  measured  in 
terms  of  our  measure  of  recruiter  effectiveness  (MORE)),  which  is  consistent  with  (and 
implied  by)  the  fact  established  immediately  above  that  a  recruiter’s  test  score  was  essentially 
not  correlated  with  his/her  performance.  Buchs  demonstrated  this  by  comparing  the  mean 
test  score  for  the  top  30  with  that  of  the  bottom  30  recruiters  (determined  by  past  recruiting 
performance)  for  each  of  the  three  categories  of  recruiters  with  different  amounts  of  recruiting 
experience  (see  Table  VI  below).  None  of  the  observed  differences  between  mean  test-score 
values  for  a  particular  category  of  recruiter  (e.g.  “Recruiters  (10-24  months)”  =  “recruiters 
with  between  10  and  24  months  of  recruiting  experience”)  of  the  top  30  recruiters  and  of  the 
bottom  30  was  statistically  significant  at  a  =  0.05  (see  Buchs  [1994,  pp.  150,  155,  and  160  of 
Appendix  H]  fbr  information  on  the  hypothesis  tests  conducted,  with  supporting  information 
on  satisfaction  of  test  assumptions  also  being  provided). 

Thus,  none  of  the  observed  differences  in  test-score  means  for  a  given  category  of  recruiter 
shown  in  Table  VI  turned  out  to  be  statistically  significant.  The  fact  that  relatively  less 
successful  recruiters  had  higher  test  scores  (i.e.  the  mean  test-score  value  for  the  bottom 
30  was  higher  than  that  for  the  top  30  recruiters)  in  the  last  two  categories  of  recruiters 
corresponds  to  the  negative  values  for  (sample)  correlation  coefficients  shown  in  Table  V 
above. 
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Table  VI.  Sales-Comprehension-Test-Score  Means  and  Standard  Deviations  for  Top  30 

and  Bottom  30  Recruiters  for  Each  of  the  Three  Basic  Categories  of  Recruiters 


Sample  Group 

Mean 

Standard 

Deviation 

Top  30  Recruiters  (0-9  months) 

(n  =  30) 

21.52 

17.47 

Bottom  30  Recruiters  (0-9  months) 

(n  =  29) 

20.00 

14.91 

Top  30  Recruiters  (10-24  months) 

(n  =  30) 

18.62 

19.82 

Bottom  30  Recruiters  (10-24  months) 
(n  =  30) 

19.57 

14.43 

Top  30  Recruiters  (>  24  months) 

(n  =  31) 

17.77 

18.08 

Bottom  30  Recruiters  (>  24  months) 
(n  =  31) 

22.71 

16.06 

3.3  Multiple- Linear-Regression  Model 

Because  Buchs  [1994]  had  shown  that  the  Sales-Comprehension-Test  (SCT)  score  of 
a  recruiter  was  by  itself  inadequate  to  predict  success,  a  multiple-linear-regression  model 
was  developed  by  Hernandez  [1994].  This  approach  considers  data  as  observations  on  a 
dependent  variable,  here  denoted  as  (DV),  corresponding  to  given  values  of  n  independent 
variables.  It  assumes  a  linear  (regression)  model  for  the  data  (here  the  subscript  i  denotes 
the  observation  and  e  denotes  an  error  term  with  certain  assumed  statistical  properties) 
and  develops  an  estimation  equation  for  the  mean  value  of  the  dependent  variable  (here  the 
measure  of  recruiter  effectiveness  (MORE))  where  jeJs  means  that  only  the  significant 

iDV)i  =  /3o  -h  f2  PAlV)ij  +  Si  (5) 

j==l 

(DV)  =  bo  +  Y.hiIV)j  (6) 

jeJs 

independent  variables  are  retained  in  the  (final)  model,  and  the  subscripted  “6s”  are  point 
estimates  (least-squares  estimators)  of  the  correspondingly  subscripted  “;0s”  (which  are  pop¬ 
ulation  parameters). 

Using  this  approach,  Hernandez  [1994,  p.  64]  developed  a  multiple-linear-regression 
model  with  the  following  estimation  equation  for  recruiter  success  in  terms  of  the 
combined  probability  (see  equation  (4))  that  the  recruiter  makes  mission  in  any 
given  month  of  the  TTE  period 
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Estimated  Mean  Success  Probability  = 


(7) 


0.54  +  0.00064  (AFQT)  +  0.00086  (SCT)  +  0.11  (Gender)  -  0.037  (PMOS) 

where  (AFQT)  denotes  the  recruiter’s  Armed-Forces-Qualification-Test  (AFQT)  score,  (SCT) 
denotes  his/her  SCT  score,  (Gender)  denotes  the  gender  variable  defined  by  equation  (3), 
and  (PMOS)  denotes  the  Primary-Military-Occupational-Specialty  (PMOS)  variable  defined 
by  equation  (1)  above.  For  simplicity,  we  will  refer  to  “Estimated  Mean  Success  Probability” 
henceforth  as  “Estimated  Success  Probability”. 

Equation  (7)  is  the  main  product  of  the  work  reported  here.  It  is  a  first  cut 
at  the  sought-after  model  to  predict  recruiter  success,  and  has  been  (partially)  scientifically 
validated.  Model  shortcomings,  however,  are  noted  below,  and  suggestions  made  (see,  espe¬ 
cially,  Section  4)  for  remedying  them.  Such  future  refinements  would  take  the  work  at  hand 
for  their  point  of  departure. 

This  model  (7)  identifies  four  recruiter  attributes  (two  represented  by  test  scores  and 
two  classification  categories)  that  can  predict  the  combined  probability  of  making  mission  in 
any  given  month  of  the  TTE  period.  Scores  on  the  AFQT  and  SCT,  respectively,  measure 
a  soldier’s  communications/organizing  and  potential  sales  skills.  The  gender  and  PMOS 
variables  are  categorical  variables  dilferentiating  groups  with  perhaps  greater  or  less  potential 
for  success  in  recruiting. 

The  numerical  sign  of  each  coefficient  in  the  model  (7)  shows  whether  the  variable  in 
question  is  positively  or  negatively  correlated  with  recruiting  success  (as  defined  above). 
Thus,  the  positive  sign  for  the  AFQT  and  SCT  scores  means  that  soldiers  with  higher  scores 
on  each  of  these  tests  tend  to  have  greater  success  as  a  recruiter.  On  the  average,  male 
recruiters  are  more  successful  than  female  recruiters  (although  this  might  be  a  statistical 
quirk  of  having  only  a  small  sample  size  for  female  recruiters'^).  A  surprising^  result  was 
that  having  a  combat-arm  PMOS  had  a  negative  effect  on  a  recruiter’s  performance. 

Based  on  their  research,  the  authors  recommend  that  the  model  (7)  be  used 
exclusively,  not  inclusively  (i.e.  be  used  to  identify  candidate  recruiters  either  to  be  elim¬ 
inated  from  recruiting  or  to  be  identified  as  a  potential  high  risk  to  be  able  to  remain  in 
recruiting  on  the  basis  of  recruiting  performance,  rather  than  be  used  blindly  to  “qualify” 
sergeants  for  recruiting).  We  are  skeptical  that  a  high  score  (i.e.  high  value  for  estimated 
success  probability  from  (7))  would  (or  even  could)  guarantee  candidate  recruiter  success, 
but  we  believe  that  a  relatively  low  predicted/estimated  success  probability  would  portend 
potential  trouble  ahead  for  the  individual.  Furthermore,  our  work  here  indicates  that  the 
model  (7)  possesses  comparable  predictive  capability  (value  for  its  coefficient  of  mul¬ 
tiple  correlation®^)  as  that  of  the  SAT  in  predicting  a  student’s  first-year-college 
performance  (i.e.  degree  of  success).  The  model’s  validity  is  further  discussed  in  Section 
3.3.2  below. 

Finally,  the  major  shortcoming  of  the  model  (7)  is  that  it  was  developed  from 
“current-employee”  (as  opposed  to  “follow-up”)  data  (see  Section  2.6  above).  No  data 
at  all  was  available  on  unsuccessful  recruiters.  It  is  most  strongly  recommended  that 
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future  work  generate  “follow-up”  data  (see  Section  2.6  above;  also  Hernandez  [1994, 
p.  69])  to  remedy  this  situation. 

3.3.1  Development  of  Model 

There  are  two  aspects  of  the  model’s  development  that  need  discussion:  concep¬ 
tual  and  statistical.  Conceptually,  one  needs  some  working  hypothesis  as  to  what  might 
be  the  significant  (in  a  physical  sense)  independent  variables  upon  which  the  dependent 
variable  (here  the  MORE)  depends.  Knowledge  of  the  prediction  of  human  performance 
by  psychologists  and  the  recruiting  system  led  us  to  consider  the  following  six  variables  as 
candidate  independent  variables: 

(1)  Sales-Comprehension-Test  (SCT)  score, 

(2)  Armed-Forces-Qualification-Test  (AFQT)  score, 

(3)  the  length  of  time  in  the  Army  before  becoming  a  recruiter  (in  months), 

(4)  the  Primary-Military-Occupational-Specialty  (PMOS)  variable  defined  by 
equation  (1), 

(5)  the  candidate-selection  variable  defined  by  equation  (2),  and 

(6)  the  gender  variable  defined  by  equation  (3). 

The  above  independent  variables  represent  an  initial  starting  point  (as  far  as  independent 
variables  are  concerned)  in  a  trial-and-error  procedure  of  investigating  which  variables  to 
include  in  the  final  model. 

A  multiple-linear-regression  model  was  then  developed  for  these  six  independent  variables 
(with  the  MORE  defined  by  (4)  being  the  dependent  variable),  and  variables  eliminated  on 
the  basis  of  statistical  analysis  that  investigated  each  variable’s  contribution  to  fitting  the 
data.  Statistical  aspects  have  to  do  with  this  analysis.  The  reason  for  differentiating  between 
the  conceptual  and  statistical  basis  for  model  development  is  to  point  out  here  that  other 
conceptual  bases  are  possible  (i.e.  other  initial  independent  variables  hypothesized  to  yield 
the  observed  values  for  the  MORE  (dependent  variable)). 
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Starting  with  the  above  initial  six  independent  variables,  Mallow’s  coefficient  (denoted 
as  Cp)  was  used  to  select  the  best  subset  of  variables  from  them  for  minimizing  the  error 
mean  square  (denoted  as  MSE)  of  the  estimated  mean  value  of  the  dependent  variable  (e.g. 
see  Hernandez  [1994,  pp.  34-37],  Fox  [1991,  pp.  18-19],  Neter,  Wasserman,  and  Whitmore 
[1993,  Chapters  18-20]).  Mallow’s  method  involves  computing  (for  each  (proper)  subset  of 
the  original  independent  variables)  the  coefficient  Cp  given  by 

Cp  =  (m  1  —  p)  {Fm-p  —  1)  +  p  (8) 

where  m  denotes  the  total  number  of  independent  variables  being  considered  for  the  model 
(here  equal  to  six),  p  denotes  one  plus  the  total  number  of  independent  variables  in  the 
subset  being  evaluated,  and  Fm-p  denotes  the  test  statistic  for  testing  the  null  hypothesis 
that  the  “/?”s  corresponding  to  the  excluded  variables  are  all  equal  to  zero. 

When  Fm-p  is  small,  the  MSE  of  the  excluded  variables  is  large  (e.g.  see  Hernandez 
[1994,  p.  34]).  When  Cp  >  p,  the  corresponding  subset  of  independent  variables  is  not  a 
good  subset  selection  because  it  has  excluded  a  variable  set  with  a  large  F-statistic- value, 
meaning  a  small  MSE  for  the  excluded  variables  (see  Hernandez  [1994,  p.  34]  for  further 
details).  Hence,  one  considers  subsets  (of  the  initial  independent  variables)  for  which  Cp  <  p 
as  candidates  for  selection  as  the  best  variable  subset. 

For  his  data  set  (see  Hernandez  [1994,  Appendix  D  (p.  97)]),  Hernandez  calculated  values 
of  Cp  for  all  62  proper  subsets  of  the  above  six  independent  variables  and  found  only  three 
values  less  than  the  corresponding  value  of  p  (see  Table  VII  below). 

Table  VH.  Candidate  Variable  Subsets  for  Selection  as  Best  Variable  Subset 
according  to  Mallow’s  Method 


Subset  Group 

Variables 
Included 
in  Model 

Value 

for 

P 

Mallow’s 

Coefficient 

Cp 

Coefficient 
of  Multiple 
Correlation 
R 

Coefficient 
of  Multiple 
Determination 

44 

ATGP 

5 

4.41 

0.3082 

0.0950 

56 

ATGS 

5 

4.71 

0.2866 

0.0822 

60 

ATGPS 

6 

5.05 

0.3317 

0.1100 

Table  VII  follows  Hernandez’s  notation,  with  the  letter  “A”  in  the  second  column  denot¬ 
ing  the  variable  “AFQT  score” ,  “T”  denoting  the  “SCT  score” ,  “G”  denoting  the  “gender 
variable”,  “P”  denoting  the  “PMOS  variable”,  and  “S”  denoting  the  “candidate-selection 
variable”.  The  coefficient  of  multiple  determination,  denoted  as  is  given  (as  usual)  by 
the  regression  sum  of  squares  divided  by  the  total  sum  of  squares  for  the  multiple-linear- 
regression  model  correspond-  ing  to  the  subset  number  under  consideration. 

The  model  based  on  subset  number  60,  which  excludes  the  length  of  time  (in  months) 
in  the  Army  before  becoming  a  recruiter  from  the  set  of  predictive  variables  (and  hence 
consists  of  the  remaining  five  independent  variables),  gives  the  best  fit  to  the  data,  since 
it  has  the  largest  value  for  the  coefficient  of  multiple  determination.  This  model  has  the 
following  estimation  equation: 
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Estimated  Success  Probability  = 

0.54  +  0.00064  (AFQT)  +  0.00086  (SCT)  (9) 

+  0.11  (Gender)  -  0.037  (PMOS)  +  0.049  (Selection) 

where  (Selection)  denotes  the  candidate-selection  variable  given  by  equation  (2)  above. 

For  the  model  corresponding  to  this  estimation  equation,  Hernandez  [1994,  pp.  39-40] 
tested  the  hypothesis^®  that  all  the  corresponding  “/l”s  were  equal  to  zero  and  found  a  p- value 
of  0.0467  for  the  F-statistic  as  used  previously.  Hence,  the  estimation  equation  (9)  is  rea¬ 
sonable  for  the  data  at  hand.  Further  hypothesis  testing  investigated  the  significance  of  the 
coefl&cients  in  (9)  and  indicated  that  the  candidate-selection  variable  had  a  zero  beta-value 
(i.e.  Ps  =  0)  in  the  regression  model  corresponding  to  the  estimation  equation(9).  Hence, 
the  term  involving  (Selection)  was  dropped  from  (9),  whence  follows  the  final  estimation 
equation  (7). 

3.3.2  Model  Validity 

What  faith  should  USAREC  have  that  the  model  developed  to  predict  recruiter  success 
(7)  will  actually  be  useful  in  the  future  for  managing  recruiters’  careers  by  estimating  this 
probability?  There  are  two  aspects  to  be  considered 

(1)  mathematical  appropriateness  of  the  model  for  describing  the  data, 

(2)  scientific  validity  that  a  theoretical  value  for  mean  combined  probability  of 
recruiter  success  will  actually  agree  with  (in  some  sense)  a  corresponding 
observed  value. 

Mathematical  appropriateness  of  the  multiple-linear-regression  model  was  investigated 
by  means  of  residual  analysis.  Independence  of  errors  (i.e.  the  “ej”s),  their  normality,  and 
linearity  of  the  model  were  investigated  and  proved  to  be  satisfactory  (Hernandez  [1994,  pp. 
45-50]).  Thus,  residual  analysis  has  validated  the  satisfaction  of  the  assumptions  underlying 
the  use  of  a  multiple-linear-regression  model.  Hence,  it  can  be  said  that  the  final  model  (7)  is 
an  appropriate  one  for  estimating  how  recruiter  success  probability  depends  on  measurable 
recruiter  attributes. 

Scientific  validation  of  the  final  model  (7)  was  much  more  difficult,  since  it  was  impossible 
to  generate  new  observed  values  of  the  combined  probability  of  recruiter  success  against  which 
to  test  (theoretical)  estimated  values  from  (7).  Consequently  data  splitting  had  to  be  resorted 
to.  The  entire  data  set  of  101  observations  was  split  into  two  separate  components:  a  model¬ 
building  subset  and  a  validation  subset.  Following  well-recognized  procedures,  Hernandez 
[1994,  pp.  50-51]  generated  a  model-building  subset  of  71  observations  and  a  validation 
subset  of  30  observations.  Model  validation  was  then  investigated  by  computing  the  mean- 
squared  predictive  residual  for  the  validation  subset  and  comparing  this  value  with  that 
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of  the  mean  squared  error  of  the  regression  model  developed  with  the  model-building  data 
subset.  Again,  Hernandez  [1994,  pp.  51-54]  reported  satisfactory  results  and  (hence)  one  can 
conclude  that  theoretical  and  observed  values  do  coincide  for  the  regression  model  developed. 
However,  it  must  always  be  born  in  mind  that  “current-employee”  data  (not  including  any 
data  for  failed  recruiters)  was  used. 

3.3.3  Cost-Benefit  Analysis  of  Proposed  Recruiter-Selection  Procedure 

It  is  of  considerable  interest  to  investigate  the  possible  savings  that  might  accrue  to 
USAREC  from  using  a  model  like  (7)  to  predict  recruiter  success  probability.  Hence,  this 
section  sketches  an  approach  for  cost-benefit  analysis  of  this  model. 

In  general,  a  cost-benefit  analysis  of  a  selection  procedure  requires  a  number  of  items  of 
information: 

(1)  the  results  of  a  regression  analysis:  in  particular  the  multiple  (or  simple) 
correlation  coeflficient  (R)  [in  this  case,  R  =  0.31], 

(2)  the  selection  ratio  (SR),  which  is  the  ratio  of  the  number  of  people  selected 
(1560  annually)  to  the  number  available  for  selection  (for  the  calculation  at  hand 
taken  to  be  6700  annually  (a  fictitious  figure  used  for  illustrative  purposes,  much 
smaller  than  the  actual  one)  [in  this  case,  SR  =  0.23], 

(3)  the  base  rate  (BR),  which  is  the  proportion  of  current  workers  who  are  suc¬ 
cessful  (not  dropouts  for  ineffectiveness)  [in  this  case  during  the  TTE  period, 
BR  =  0.755], 

(4)  the  cost  of  a  selectee,  including  travel  and  schooling  [in  this  case,  $12,287.51]. 

The  analysis  makes  use  of  the  Taylor-Russell  tables  (Taylor  and  Russell  [1939];  see  also 
Hernandez  [1994,  pp.  72-74]),  which  use  R,  SR,  and  BR  to  yield  an  estimate  of  the  propor¬ 
tion  of  selectees  expected  to  be  successful  as  a  result  of  the  use  of  the  selection  procedure 
associated  with  the  value  R  (the  expected  success  rate  (ESR))  [in  this  case,  ESR  = 
0.90]. 

Two  steps  remain;  the  first  is  to  convert  BR  and  ESR  to  their  complements:  failure  rates 
[in  this  case,  0.245  and  0.10,  respectively].  The  final  step  is  to  multiply  each  of  these  failure 
rates  by  the  product  of  1560  and  $12,287.51  to  obtain  the  (total)  cost  of  failures  occurring 
without  and  with  the  use  of  the  selection  procedure  [in  this  case, 

(COST  WITHOUT)  =  0.245  x  1560  x  $12,287.51  =  $4,696,286.30 
(COST  WITH)  =  0.100  X  1560  x  $12,287.51  =  $1,916,851.50 

The  difference  between  these  two  dollar  amounts  is  the  annual  savings  (exclusive  of  the 
cost  of  testing  the  candidates)  [in  this  case,  annual  savings  =  $2,779,434.80].  This  is  a 
conservative  figure  because  SR  is  likely  to  be  much  smaller  than  0.23  and  also  because 
the  study  at  hand  used  the  current-employee  method  rather  than  the  follow-up  method  to 
determine  R. 
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Other  cost-benefit  analyses  appear  in  Buchs  [1994,  pp.  59-65]  and  Hernandez  [1994,  pp. 
57-61].  All  such  analyses  suggest  quite  substantial  potential  savings  for  USAREC  from  use 
of  a  model  like  (7).  We  have  barely  scratched  the  surface  here,  and  much  more  important 
work  remains  to  be  done. 


4  Suggested  Future  Work 


Prospects  for  the  future  are  extremely  bright  provided  that  efforts  are  channeled 
in  those  directions  that  are  most  useful  for  USAREC.  For  example,  results  of  our  initial 
work  reported  here  (see  Section  3  above)  indicate  that  the  only  way  to  validate  the 
“Sales-Comprehension”  Test  (Bruce  [1988])  is  to  use  the  follow-up  (or  predictive) 
method  of  testing:  i.e.  to  test  candidate  recruiters  before  they  enter  into  the 
recruiting  program  and  then  to  monitor  their  subsequent  progress  in  Recruiter 
School,  during  the  TTE  period,  and  then  during  their  normal  recruiting  tour  of  duty.  This 
is  also  then  essentially  the  only  way  to  develop  a  valid  recruiter-selection  model. 
Moreover,  follow-up  testing®^  should  be  pursued  and  related  to  the  recruiter’s  progress. 
Finally,  if  USAREC  adopts  station  missioning,  then  the  methodology  developed  here  should 
be  adapted  to  this  case. 

Thus,  future  work  should  focus  on  the  following  three  themes: 

(1)  pilot  program  (within  USAREC)  of  testing  (primarily  using  the  “Sales- 
Comprehension”  Test,  but  also  considering  interests  and  communica¬ 
tions  and  organizing  skills)  of  recruiters  before  they  enter  Recruiter  School 
and  during  their  subsequent  management; 

(2)  adaptation  of  the  methodology  developed  here  to  station  missioning 
(including  the  above  pilot  program); 

(3)  stress  management  (primarily  based  on  stress-related  inputs  (especially 
stress  tolerance) from  the  Minnesota  Multi-phasic  Personality  Inventory 
(2nd  version)(MMPI-2)). 

These  three  approaches  in  some  sense  are  interrelated,  since  the  choice  of  the  Measure  of 
Recruiter  Effectiveness  (MORE)  (e.g.  station  missioning)  can  be  changed  to  reduce  the 
stress  level  for  an  individual  recruiter  without  (hopefully)  compromising  the  achievement  of 
overall  USAREC  goals.  Additionally,  selecting  recruiters  with  greater  sales  aptitude  should 
also  reduce  the  stress  level  for  recruiters. 

Additional  topics  to  be  considered  under  the  three  above  themes  (or  otherwise)  are  the 
following: 

(1)  stress-tolerance  measurement  and  stress  management, 

(2)  development  of  measures  of  recruiter  effectiveness  for  station  mis¬ 
sioning  (i.e.  the  yardstick  for  evaluating  recruiter  performance  will  have  to 
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be  changed  for  station  missioning). 

(3)  revising  the  Sales-Comprehension  Test  to  be  more  sensitive  to 
USAREC  requirements, 

(4)  measurement  of  other  attributes  of  candidate  recruiters  (e.g.  interest 
in  sales,  communication  ability,  organizational  ability), 

(5)  diagnostics  for  remedial  training  of  recruiters,  and 

(6)  data  entry  and  data  management  within  USAREC. 


5  Final  Comments 

Our  initial  results  concerning  the  development  of  a  multiple-linear-regression  model  to 
predict  candidate-recruiter  success  in  Recruiter  School  and  during  subsequent  duties  have 
been  very  encouraging.  Literature  research  revealed  that  aptitude  testing  had  produced 
minimum-qualifying  scores  for  entry  into  every  U.S.  Army  MOS  except  Recruiting.  Conse¬ 
quently,  a  current-employee/ concurrent  testing  program  with  a  Sales-Comprehension  Test 
was  under-taken.  This  was  the  first  use  of  such  a  sales-aptitude  test  to  assist  in  the  selec¬ 
tion  of  recruiters  by  any  U.S.  Service.  Currently,  the  USAF  Recruiting  Service  (which  was 
presented  the  results  of  our  research  at  a  Joint  Recruiting  Conference)  is  planning  to  use  the 
Sales-Comprehension  Test  to  improve  its  process  of  selecting  and  training  recruiters. 

Lack  of  sufficient  time,  however,  precluded  pursuing  a  “follow-up” /prediction  testing  pro¬ 
gram,  and  hence  only  successful  recruiters  were  available  for  testing.  Nevertheless,  prospects 
for  developing  such  a  model  appear  to  be  very  favorable. 
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FOOTNOTES 


1.  The  discussion  of  “Problem  Statement”  in  the  main  text  follows  that  of  the  original 
proposal  to  USAREC  for  the  work  reported  here,  except  that  the  discussion  has  been 
somewhat  condensed. 

2.  The  exact  number  of  aptitude  tests  (e.g.  see  Weitzman  [1985])  has  changed  (and 
undoubted  will  continue  to  change  over  time)  (Weitzman  [1994b|). 

3.  As  estimated  by  Weitzman  [1994b]. 

4.  Technically  speaking,  the  recruiting  MOS  refers  to  those  Regular  Army  (RA)  recruiters 
who  are  “permanent”  recruiters  (i.e.  they  will  remain  recruiters  until  they  retire  from 
the  U.S.  Army).  A  recruiter  can  only  achieve  this  status  as  a  result  of  superior  per¬ 
formance  in  (among  other  things)  meeting  recruiting  quotas  (see  Section  1.4  above). 
Hence,  it  is  probably  unheard  of  for  such  a  recruiter  not  to  make  mission.  Moreover, 
“temporary”  recruiting  sergeants  (i.e.  those  who  are  not  “permanent”  recruiters)  are 
assigned  for  a  three-year  tour-of-duty  as  a  recruiter.  It  is  those  “temporary”  recruiters 
who  fail  to  meet  their  quotas  (see  Section  1.4  above)  that  are  of  concern  to  USAREC. 

5.  The  fact  that  sales  aptitude  is  considered  to  be  a  necessary  ingredient  for  successful 
recruiting  by  USAREC  Manual  100-5  [1989]  and  no  sales-aptitude  test  is  included  in 
the  ASVAB  (or  given  as  a  later  screening  test)  is  one  of  the  major  initial  findings  of 
the  work  at  hand.  Informal  discussions  with  recruiting  sergeants  at  the  Salinas  U.S. 
Army  Recruiting  Station  substantiated  the  importance  of  sales  aptitude. 

6.  Martin  M.  Bruce’s  Sales  Comprehension  Test  goes  back  at  least  to  the  early  1950s 
and  has  evolved  through  several  different  versions.  The  current  version  (Bruce  [1988]) 
was  developed  (with  essentially  no  impact  on  the  content  of  the  examiner’s  manual) 
after  the  first  publication  of  the  examiner’s  manual  (Bruce  [1978]).  Supplements  to  this 
edition  of  the  examiner’s  manual  have  subsequently  appeared  (Bruce  [1994])  (updating, 
for  example,  validity  studies).  Thus,  the  examiner’s  manual  actually  bears  an  earlier 
publication  date  than  the  Sales  Comprehension  Test  (Bruce  [1988]). 

7.  Bruce  [1994]  has  kindly  pointed  out  that  his  Sales  Comprehension  Test  is  by  far  the 
most  widely  used.  Over  one  million  tests  have  been  administered.  The  test  has  been 
translated  into  many  languages,  including  French,  Polish,  Danish,  German,  and  Span¬ 
ish.  It  literally  has  been  given  throughout  the  civilized  world  (Bruce  [1994]).  Moreover, 
he  says  that  such  research  on  sales-comprehension  testing  goes  back  to  1928.  Prior  to 
using  sales-comprehension  tests  in  the  process  of  selecting  salesmen,  insurance  compa¬ 
nies  used  essentially  biographic  data.  This  method  was  producing  less-than-satisfactory 
results.  Consequently,  the  first  sales-comprehension  test  was  developed  by  a  staff  of 
psychologists  of  an  insurance  company  in  order  to  more  effectively  select  salesmen 
(Bruce  [1994]). 
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8.  George  Germadnik  [1994],  Chief  of  the  Market  Research  Section  (Headquarters  USAF 
Recruiting  Services),  has  generously  supplied  (informally)  the  authors  with  much  valu¬ 
able  background  information  concerning  USAF  recruiting  operations.  For  example, 
100%  of  USAF  recruiters  are  volunteers,  and  it  does  have  a  “recruit-the-recruiter”  pro¬ 
gram.  The  occurrence  of  “relief  action”  for  a  recruiter  has  not  been  so  acute  as  for  the 
U.S.  Army,  although  “spikes”  in  such  occurrences  do  occasionally  occur.  Moreover, 
CPT  Andy  Hernandez  was  invited  to  make  a  presentation  of  his/our  results  to  a  Joint 
Recruiting  Conference  hosted  by  USAREC  in  June  1994,  and  George  Germadnik  at¬ 
tended  this  conference.  He  has  kindly  provided  the  authors  with  useful  information 
several  times  since  then. 

9.  These  seventeen  time  periods  are  known  throughout  the  recruiting  community  as  the 
“seventeen  races.”  Further  background  information  on  them  are  given  in  Hernandez 
[1994,  p.  13]. 

10.  This  point  was  revealed  during  research  discussions  with  USAREC  representatives  on 
Wednesday,  August  31,  1994. 

11.  There  were  four  commercially-available  tests  considered  (Weitzman  [1994a])  for  this 
pilot  study: 

(1)  “Sales-Aptitude  Test:  ETSA  Test  7A,”  Employer’s  Test  and  Services 

Associates,  Inc., 

(2)  “Sales  Aptitude  Check  List,”  Science  Research  Associates,  Inc., 

(3)  “Sales  Motivation  Inventory,  Revised,”  Martin  M.  Bruce,  Ph.D.,  Pub¬ 

lishers, 

(4)  “Sales-Comprehension  Test,  Form  M,  Revised,”  Martin  M.  Bruce,  Ph.D., 
Publishers. 

12.  In  all,  four  commercially-available  tests  were  considered.  See  Footnote  11  for  further 
details. 

13.  See  Footnote  7. 

14.  As  pointed  out  by  Bruce  (see  Footnote  7),  research  on  sales- aptitude  testing  apparently 
goes  back  to  1928  and  originated  then  in  the  insurance  industry.  The  origin  of  the  test 
under  immediate  consideration  (“Sales  Comprehension  Test,  Form  M,  Revised”  (Bruce 
[1988]))  can  be  traced  back  to  1946  (Bruce  [1978,  p.  3]).  A  discussion  of  the  evolution  of 
the  test  (focusing  primarily  on  design  considerations)  from  its  1946  origins  (up  through 
a  1971  major  revision)  is  also  contained  in  Bruce  [1978,  p.  3]. 
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15.  The  AFQT  (e.g.  see  Weitzman  [1985])  is  composed  of  four  parts  (a  subset  of  the 
ASVAB):  the  word-knowledge,  paragraph-comprehension,  arithmetic-reasoning,  and 
mathematical-knowledge  tests.  The  first  two  parts  measure  verbal  ability  (i.e.  a  sol¬ 
dier’s  ability  to  extract  and  analyze  information  from  written  sources),  while  the  latter 
two  parts  measure  quantitative  ability  (Eitelberg  [1988,  pp.  68-74]). 

16.  Although  operations  research  (OR)  historically  arose  out  of  the  necessity  to  evaluate 
the  effectiveness  of  new  t3q)es  of  military  systems/operations  during  World  War  II  (see 
Morse  and  Kimball  [1951,  Chapter  3];  also  McCue  [1990]),  a  rather  cursory  examina¬ 
tion  of  major  OR  textbooks  (e.g.  Hillier  and  Lieberman  [1967],  Wagner  [1969])  reveals 
(with  one  notable  exception  (Naval  Operations  Analysis  (NOA)  [1977]))  absolutely  no 
treatment  of  system  effectiveness/development  of  measures  of  effectiveness  (MOEs) 
for  any  type  of  system,  much  less  for  military  systems.  This  is  indeed  ironic,  since 
not  only  did  Morse  and  Kimball  [1951,  Chapter  3]  in  their  classic  and  seminal  work 
Methods  of  Operations  Research  devote  an  entire  chapter  to  “The  Use  of  Measures  of 
Effectiveness,”  but  also  Herrmann  and  Magee  [1953]  have  called  the  measure  of  effec¬ 
tiveness  (MOE)  one  of  the  “four  concepts  of  fundamental  importance  to  the  practice 
of  OR.” 

Any  DoD  practitioner  involved  with  evaluating  military  systems  can  attest  to  the 
essential  nature  of  understanding  the  basic  concepts  concerning  system  effectiveness. 
The  simple  facts,  unfortunately,  are  that 

(1)  academicians  in  civilian  colleges  and  universities  essentially  have 
no  interest  in  system  effectiveness,  and 

(2)  DoD  has  done  a  poor  job  of  institutionalizing  knowledge  about 
this  critical  topic  (see  Taylor  [1983,  Preface]  for  further  discussion  of 
DoD  problems  of  institutionalizing  such  knowledge). 

Consequently,  one  of  the  authors  here  (Taylor)  has  had  to  develop  his  own  theory  of  the 
measurement  of  system  effectiveness,  which  unfortunately  remains  largely  unpublished. 
This  theory  is  based  on  viewing  an  MOE  as  a  decision  aid  (within  the  context  of 
a  specific  decision  problem)  and  considering  MOE  development  to  be  based  on  the 
following  four  fundamental  concepts: 

(1)  effectiveness, 

(2)  decision  context, 

(3)  hierarchy  of  systems, 

(4)  measurement. 

Consideration  of  the  decision  context  leads  one  to  consider  the  following  aspects: 
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(1)  problem  statement  (question), 

(2)  organizational  level  of  decision  maker  erspective  of  decision 
maker/analyst), 

(3)  system  (use  systems  approach:  identify  system,  supersystem, 
subsystems), 

(4)  alternatives, 

(5)  nature  of  planning  horizon  (i.e.  short-term  or  long-term), 

(6)  evaluation/forecasting/modeling  approach, 

(7)  threat  and  the  system’s  operational  environment, 

(8)  system  effectiveness, 

(a)  qualitative  effectiveness  concept, 

(b)  quantification  of  effectiveness, 

(i)  “combat  outcome”  MOE, 

(ii)  surrogate  MOE. 

17.  The  distinction  between  performance  and  effectiveness  is  fundamental,  but  apparently 
poorly  understood  by  the  operations-research  community.  Correspondingly,  one  refers 
to  the  corresponding  quantifications  of  these  concepts  as  a  measure  of  performance 
(MOP)  and  a  measure  of  effectiveness  (MOE).  According  to  Taylor’s  theory  of  the 
measurement  of  system  effectiveness  (which  is  based  on  considering  an  MOE  to  be  a 
decision  aid  and  then  on  considering  the  decision  context),  MOPs  refer  to  a  system’s 
inputs  and  essentially  describe  the  system’s  capabilities,  while  MOEs  refer  to  its  output 
(and  how  this  output  measures  up  to  the  system’s  goals). 

Moreover,  the  military  is  composed  of  hierarchical  systems  for  span-of-control  reasons 
(cf.  Wohl  [1981],  Lomov  [1983,  pp.  186-189]).  Moiseyev  [1975,  p.  465]  points  out  that 
hierarchical  structure  is  created  for  complex  systems  as  a  means  for  human  beings  to 
cope  with  their  complexity.  Such  coping  with  complexity  is  caused  by  the  limits  of 
human  information  processing  (Miller  [1956]).  Irrespective  of  any  theoretical  justifica¬ 
tion,  hierarchies  of  military  systems  (e.g.  see  Rudwick  [1969,  p.  52])  have  been  created 
by  human  trial  and  error  over  the  course  of  history.  Here  (as  is  usually  the  case  (cf. 
Rudwick  [1969,  pp.  52-53]))  a  system  is  defined  as  a  set  of  elements  unified  in  the 
achievement  of  a  goal  (or  goals)  (and  thus  behaving  as  a  single  entity).  Corresponding 
to  a  hierarchy  of  systems  is  then  a  hierarchy  of  goals  that  must  be  considered  for  de¬ 
veloping  MOEs  (which  accordingly  also  appear  as  a  hierarchy  of  MOEs).  Moreover,  as 
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pointed  out  by  the  Soviet  systems  analyst  Modin  [1981,  p.  8]  (very  loose  translation 
given  here), 

In  a  hierarchy  of  systems,  the  concept  of  goal  is  relative:  that  which  is  a 
goal  (i.e.  target  for  output)  at  one  level  is  at  the  same  time  the  means  (i.e. 
input)  for  attaining  the  goal  of  the  next  higher-level  system. 

Thus,  the  same  quantity  paradoxically  can  simultaneously  be  both  an  MOP  and  an 
MOE,  and  hence  system  goals  must  be  clearly  understood.  Such  considerations  were 
used  in  the  development  of  the  MORE  used  in  the  work  at  hand. 

18.  Before  Hernandez  [1994,  pp.  12-15]  clearly  stated  our  final  perception  of  USAREC’s 
goal  for  a  recruiter  (from  which  the  choice  of  MORE  unambiguously  follows),  Buchs 
[1994,  pp.  21-27]  also  had  considered  another  MORE:  cumulative  percent  success  = 
lOOx  (total  enlistees  brought  into  Army  for  given  period  of  time) / (total  quota  of  enlis¬ 
tees  to  be  recruited  for  this  period).  Buchs’s  [1994,  pp.  50-68]  conclusions  essentially 
did  not  depend  on  the  MORE  chosen. 

19.  We  have  used  here  terminology  current  in  the  field  of  personnel  selection  and  manage¬ 
ment  (e.g.  see  McKenna  [1967]).  Atwater  etal.  [1986]  use  the  terminology  “concurrent 
method.” 

20.  Atwater  et  al.  [1986]  use  the  terminology  “predictive  method.”  See  also  Footnote  19 
above. 

21.  Here  we  are  following  the  terminology  of  Neter,  Wasserman,  and  Whitmore  [1993,  p. 
240]  to  denote  the  fact  that  judgment  was  used  to  infer  that,  for  example,  the  Balti¬ 
more  Battalion  of  the  First  Recruiting  Brigade  is  representative  of  an  “outstanding” 
recruiting  battalion. 

22.  The  battalions  in  question  were  the  Baltimore  Battalion  of  the  First  Recruiting 
Brigade  and  also  the  Santa  Ana  Battalion  of  the  Sixth  Recruiting  Brigade. 

23.  Here  we  are  essentially  using  Neter,  Wasserman,  and  Whitmore’s  [1993,  p.  240]  defini¬ 
tion  of  judgment  sample:  A  sample  for  which  judgment  is  used  to  select  representative 
elements  from  the  population  or  to  infer  that  it  is  representative  of  the  population  is  a 
judgment  sample.  See  Footnote  21  above  and  also  Neter,  Wasserman,  and  Whitmore 
[1993,  p.  240]  for  further  information. 

24.  The  control  group  consisted  of  all  attendees  of  a  Basic  NCO  Course  (BNCOC)  held  at 
Fort  Knox,  Kentucky  in  February  1994. 
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25.  The  abbreviation  “TTE”  stands  for  “Transitional  Training  and  Evaluation.”  The  so- 
called  TTE  Program  follows  graduation  from  the  Army  Recruiting  Course  (ARC)  and 
essentially  provides  transition  to  field-force  recruiting  (see  USAREC  [1989,  p.l40]  for 
further  details;  also  Appendix  A).  The  term  “TTE  period”  then  refers  to  the  initial 
nine  months  of  the  36-month  recruiting  tour  of  duty  after  graduation  from  the  ARC 
(see  Appendix  A). 

26.  Hernandez  [1994,  Appendix  A  (pp.  75-91)]  actually  gives  the  entire  database  (for  all 
276  recruiters)  that  was  used  by  Buchs  [1994]  for  most  of  his  analyses  (except  those 
involving  the  BNCOC  data).  With  this  database,  these  analyses  of  Buchs  can  be 
recreated. 

27.  So  far,  no  test  score  has  been  identified  as  belonging  to  a  recruiter  classified  as  “un¬ 
successful”  (see  Appendix  A  for  a  definition  of  terms)  by  USAREC. 

28.  Some  of  the  statistical-analysis  results  given  in  Section  3.1  were  developed  specifically 
for  the  writing  of  this  report.  The  origin  of  these  results  will  not  be  specifically  iden¬ 
tified,  but  some  key  aspects  will  be  highlighted  in  footnotes. 

29.  As  USAREC  policy  regarding  recruiter  management  (i.e.  the  mission  [goals]  assigned 
to  recruiters)  changes,  the  measure  of  recruiter  effectiveness  (MORE)  may  have  to 
be  changed.  In  particular,  station  missioning  may  well  require  a  different  MORE  for 
evaluating  recruiter  performance. 

30.  Buchs  [1994]  also  reports  P-values  (e.g.  see  Weiss  and  Hassett  [1991,  pp.  449-450]) 
(observed  significance  levels)  for  essentially  all  hypothesis  tests. 

31.  Normality  of  the  underlying  populations  from  which  samples  were  drawn  has  essen¬ 

tially  been  established  by  Buchs  [1994].  Equality  of  normal-population  variances  for 
recruiters  and  non-recruiters  was  investigated  with  a  two-sided  F-test  (e.g.  see  Neter, 
Wasserman,  and  Whitmore  [1993,  pp.  417-421]).  Denoting  non-recruiters  as  popu¬ 
lation  1  and  recruiters  as  2  and  using  a  =  0.05,  we  compute  the  observed  value  of 
the  test  statistic  F^st  -  {sj =  1.95  >  =  1-4761,  where  {F^tT^^ 

denotes  the  upper  critical  value  of  the  test  statistic  for  a  two-sided  test.  Hence,  the 
null  hypothesis  (equality  of  variances)  is  rejected  at  a  significance  level  =  0.05.  Also, 
the  observed  value  of  the  test  statistic  corresponds  to  a  P-value  (observed  significance 
level)  equal  to  0.0006,  which  (as  it  should  be)  is  less  than  the  significance  levela. 
Investigating  equality  of  means  for  large  samples  from  arbitrary  (i.e.  variances  not 
necessarily  equal)  populations,  we  compute  the  observed  value  of  the  test  statistic 
Ztest  =  (-^1  —  X2)ls{X\  —  X 2}  =  6.32  >  (zcrit)“^^  =  1-960,  where  s{X}  denotes  the 
estimated  standard  error  of  the  mean.  This  result  is  extremely  highly  statistically  sig¬ 
nificant,  corresponding  to  a  P-value  <  0.000001.  Hence,  the  null  hypothesis  of  equality 
of  means  is  handily  rejected  (as  even  casual  inspection  of  Table  IV  indicates  that  it 
should). 
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32.  Here  we  mean  testing  at  additional,  subsequent,  selected  points  in  time  during  the 
career  of  a  recruiter. 

33.  See  Hernandez  [1994,  p.  65]. 

34.  In  many  discussions  with  recruiting  leaders,  the  point  emerged  frequently  that  soldiers 
with  combat-arms  PMOSs  had  a  tendency  to  be  more  mission-oriented  than  those  not 
directly  involved,  with  combat  in  their  jobs  and  therefore  had  greater  success  in  making 
mission  as  a  recruiter. 

35.  See,  for  example,  Neter,  Wasserman,  and  Whitmore  [1993,  pp.  606-607]  for  a  discussion 
of  the  coefficient  of  multiple  correlation.  The  coefficient  of  simple  correlation 
is  discussed  on  pp.  557-558  of  this  text  (see  also  Weiss  and  Hasset  [1991,  p.  622]). 

36.  For  all  Hernandez’s  [1994]  significance  tests,  a  significance  level  of  a  =  0.10  was  used. 

37.  See  Footnote  32  above. 
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Appendix  A:  Systems  Analysis  for  Recruiter 

Selection  and  Training 

The  U.S.  Army  Recruiting  Command  (USAREC)  meets  its  mission  requirements  to 
provide  manpower  for  active  Army  units  by  operating  in  a  mode  unique  to  Army  Major 
Commands  (MACOMs).  Key  to  the  successful  accomplishment  of  its  mission  is  the  selec¬ 
tion  and  training  of  capable  recruiters.  This  Appendix  outlines  this  selection  and  training 
process  and  concludes  by  suggesting  strategies  for  implementing  the  Sales-Comprehension 
Test  within  that  process.  Figures  1  and  2  below  provide  a  visual  guide  for  the  following 
paragraphs. 

The  majority  of  recruiters  are  selected  from  enlisted  non-commissioned  officers  (NCOs)  in 
the  grade  of  E5  and  E6.  These  soldiers  serve  in  essentially  all  Military  Occupational  Special¬ 
ties  (MOSs)  and  are  selected  in  accordance  with  Army  Regulation  (AR)  601-1,  Assignment 
of  Enlisted  Personnel  to  the  US  Army  Recruiting  Command.  This  regulation  provides  an 
initial  screening  for  potential  recruiters  and  eliminates  individuals  with  obvious  undesirable 
characteristics  such  as  financial  hardship  or  educational  deficiencies.  A  soldier  meeting  the 
601-1  criteria  can  volunteer  for  recruiting  duty.  There  are  a  certain  number  of  incentives  for 
volunteering  such  as  accelerated  promotions  and  preference  for  location  of  assignment.  Since 
these  individuals  are  motivated  for  their  future  recruiting  duties,  USAREC  desires  that  the 
majority  of  future  recruiters  are  volunteers.  Although  the  number  is  increasing,  only  10%  of 
the  recruiter  field  force  enters  in  this  manner.  The  remaining  90%  are  nominated  by  their 
particular  career  branch.  This  is  a  personnel  decision  where  the  Army’s  Personnel  Command 
makes  the  selection  based  upon  the  AR  601-1  guidance  and  the  availability  of  soldiers  for  re¬ 
assignment  to  USAREC.  Because  duty  as  a  recruiter  is  not  considered  a  “good”  assignment, 
soldier  motivation  greatly  varies.  AR  601-1  selection  criteria  are  presented  in  Appendix  B. 

Regardless  whether  the  soldier  volunteered  or  was  nominated,  AR  601-1  requires  that 
the  soldier  be  given  a  personal  interview  with  the  first  Lieutenant  Colonel  (LTC)  in  the 
soldier’s  chain  of  command.  The  purpose  of  this  interview  is  to  further  eliminate  soldiers 
who  may  be  inappropriate  for  recruiting  duty.  The  interview  requires  the  LTC  to  insure 
that  all  the  specified  AR  601-1  criteria  are  met  and  then  further  examines  the  soldier’s 
motivational  and  interpersonal  communicative  skills.  The  hope  is  that  this  mid-level  manager 
has  the  balance  between  understanding  the  Army’s  needs  for  quality  recruiters  and  personal 
knowledge  of  that  particular  soldier’s  potential.  This  interview  is  not  determinative  in  itself, 
but  rather  is  forwarded  along  with  the  soldier’s  application  package  to  the  Army  Personnel 
Command  (PERSCOM).  However,  it  would  be  very  unlikely  that  a  soldier  would  be  selected 
for  recruiting  duty  contrary  to  the  LTC’s  recommendation.  PERSCOM  then  makes  the  final 
selection  of  potential  recruiters  in  accordance  to  the  needs  of  the  Army. 

All  soldiers  who  volunteered  and  those  who  were  nominated  must  attend  the  Army  Re¬ 
cruiting  Course  (ARC).  This  is  an  extensive  six  week  training  period  presently  conducted 
at  Fort  Benjamin  Harrison,  Indiana  (ARC  will  move  to  Fort  Jackson,  South  Carolina  in 
early  1995).  The  school  serves  as  a  transition  from  the  soldier’s  previous  MOS  training  and 
provides  him  with  the  initial  background  necessary  to  become  a  successful  recruiter.  About 
3%  of  the  soldiers  do  not  complete  the  course  and  are  reassigned  outside  of  USAREC.  It  is 
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worth  noting  that  ARC  is  operated  by  Training  and  Doctrine  Command  (TRADOC)  not 
by  USAREC.  There  is  a  close  association  between  ARC  and  USAREC  as  all  the  instructors 
have  previously  been  successful  recruiters  and  there  is  close  coordination  with  USAREC  as 
to  the  instructional  content  of  ARC.  Upon  graduation  from  ARC  the  recruiter  is  awarded 
the  Recruiter  Badge  prior  to  initial  assignment  as  a  new  recruiter. 

A  new  recruiter  is  detailed  to  a  36  month  period  in  USAREC.  The  initial  nine  months  is 
designated  as  the  Transitional  Training  and  Evaluation  (TTE)  period.  TTE  represents  an 
intensive  training  period  intended  to  supplement  the  training  from  ARC.  During  the  TTE 
a  soldier  can  also  be  found  to  be  unsuitable  for  recruiter  duty  in  which  case  he  would  be 
reassigned  to  duties  within  his  MOS.  Approximately  4%  attrite  from  the  recruiter  pool  (all 
recruiters)  in  this  manner. 

After  completion  of  TTE,  the  soldier  becomes  a  field  force  recruiter  for  the  remaining  27 
months  in  USAREC.  A  recruiter  can  be  eliminated  from  the  field  force  by  three  methods 
as  specified  by  AR  601-1.  A  recruiter  can  be  relieved  without  prejudice.  This  reassignment 
is  not  considered  to  have  an  adverse  career  impact  as  the  other  two  methods.  This  cate¬ 
gory  of  elimination  is  intended  for  personnel  unable  to  meet  the  required  recruiting  criteria 
specified  in  AR  601-1  through  no  fault  of  their  own.  Such  things  as  recent  physical  or  med¬ 
ical  limitations,  an  adverse  change  in  financial  status,  or  a  change  in  the  soldier’s  family 
situation  are  categorized  into  this  type  of  separation  action.  A  recruiter  can  be  reassigned 
due  to  ineffectiveness.  A  soldier  would  be  placed  in  this  category  if  he  failed  to  respond  to 
training,  performance  counseling  and  other  guidance.  Ineffective  relief  candidates  require 
sales  technique  training  and  documented  counseling  before  the  relief  can  be  substantiated. 
Additionally,  a  recruiter  can  be  relieved  for  ineffectiveness  due  to  failure  to  meet  assigned 
production  standards  or  necessary  sales  ability.  Prior  to  relief  the  soldier  must  have  been 
given  remedial  training  and  counseling.  AR  601-1  provisions  allow  that  this  means  of  re¬ 
lief  can  also  be  used  for  recruiters  who  fail  to  attain  an  adequate  knowledge  of  regulations, 
programs,  policies  and  procedures  related  to  recruiting  duty  or  for  a  failure  to  adequately 
train  subordinates.  The  final  method  for  relief  is  for  unsuitability.  This  category  includes 
those  recruiters  who  committed  an  illegal  recruiting  practice,  breached  a  civil  law,  failed  to 
pay  just  debts  or  failed  to  maintain  acceptable  standards  of  appearance  (including  physical 
fitness).  The  numbers  of  reliefs  for  FY  1990  to  FY  1992  compared  to  the  total  assigned 
strength  is  given  in  Table  A-I  below. 

At  the  completion  of  the  36  month  assignment  a  recruiter  has  the  opportunity  to  select 
the  OOR  MOS  and  remain  a  recruiter  for  subsequent  assignments.  If  he  chooses  not  to 
remain  as  a  recruiter,  the  soldier  will  be  reassigned  in  his  MOS.  A  very  real  concern,  but  one 
not  addressed  in  this  report  is  a  soldier  that  once  he  leaves  USAREC  is  no  longer  competitive 
for  retention  in  the  Army  due  to  a  “bad”  eflaciency  rating  while  in  USAREC. 
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Table  A-I.  Number  of  Recruiter  Reliefs  by  Category  for 
USAREC  in  FY  1990  to  FY  1992 


Category  of  Relief 

FY1990 

FY  1991 

FY1992 

New  Recruiter 
(TTE-Period  Failure) 

350 

336 

158 

Ineffectiveness 

243 

148 

51 

Unsuitability 

104 

152 

57 

Unqualified 

(Relieved  without  Prejudice) 

79 

66 

43 

Total  Number  of  Reliefs 

776 

702 

309 

Total  USAREC  Assigned 
Strength 

7639 

7006 

6576 

There  are  several  strategies  for  using  the  Sales-Comprehension  Test  described  in  the  main 
body  of  this  report.  The  test  could  be  given  by  the  “Recruite  the  Recruiter”  Team.  The 
test  is  easy  to  administer  takes  a  short  time  (approximately  20  minutes)  to  complete,  and 
can  be  graded  quickly.  The  test  could  be  given  as  soldiers  waited  for  their  interviews  with 
the  Recruiter  team.  The  results  would  give  the  recruiter  immediate  information  that  could 
substantiate  the  “gut  feel”  about  the  soldier  from  the  interview.  Certain  borderline  cases 
might  be  resolved  with  this  additional  information.  Another  possible  strategy  would  be  to 
give  the  test  to  all  Basic  NCO  Courses  (BNOCs).  This  military  school  is  composed  of  soldiers 
in  the  E5  pool  and  giving  the  test  there  could  forecast  soldiers  with  a  potential  to  be  good 
recruiters.  These  soldiers  could  be  identified  for  the  Recruit  the  Recruiter  team  as  “high 
potential”  candidates.  Finally,  the  test  could  be  administered  as  part  of  the  inprocessing  at 
ARC.  Soldiers  with  low  scores  could  be  given  more  intensive  training  or  be  reassigned. 
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Appendix  B:  Recruiter  Selection  Criteria 

To  qualify  for  selection  as  a  US  Army  recruiter,  either  as  a  volunteer  or  as  a  DA  selected 
recruiter,  a  soldier  must: 

1.  Be  a  citizen  of  the  United  States. 

2.  Be  a  high  school  graduate  with  diploma  or  have  1  year  of  college  with  a  high  school 
General  Education  Development  (GED)  (No  waiver).  College  Level  Entrance  Program 
(CLEP),  Department  of  the  Army  Non-Resident  Testing  Education  System  (DANTES) 
and  military  service  credit  do  not  apply. 

3.  Have  a  minimum  GT  score  of  110  waivable  to  100  with  an  ST  score  of  100. 

4.  Be  at  least  21  years  old,  but  not  more  than  35  years  old  at  time  of  selection. 

5.  Be  a  SGT,  SSG,  or  SEC.  (A  SFC  may  not  have  more  than  2  years  time  in  grade  at 
the  time  of  selection).  SSG(P)  or  SFC  must  be  an  Advanced  Noncommissioned  Officer 
Course  (NCO)  graduate  (No  waiver)).  (A  SSG  must  be  a  COC  graduate).  (A  SGT 
must  be  a  Primary  Leadership  Development  Course  (PLDC)  graduate  (No  waiver)). 

6.  Have  no  less  than  4  years  time  in  service  and  no  more  than  8  years  time  in  service  if 
a  SGT  no  more  than  12  years  time  in  service  if  a  SSG,  or  no  more  than  14  years  time 
in  service  if  a  SFC. 

7.  Have  completed  1  year  of  service  since  reclassification  per  AR  600-200. 

8.  Not  be  currently  assigned  to  the  Military  Entrance  Processing  Command  (MEPCOM). 

9.  Meet  the  height  and  weight  standards  of  AR  600-9  or  possess  a  medical  determination 
of  acceptable  body  fat  limits  (No  waiver). 

10.  Have  a  minimum  physical  profile  of  13221.  (No  shaving  profile). 

11.  Have  no  lost  time  during  the  current  enlistment  or  in  the  past  3  years,  whichever  is 
longer  (No  waiver). 

12.  Possess  or  be  able  to  obtain  a  valid  driver’s  hcense.  Assignment  as  a  recruiter  involves 
an  extensive  amount  of  automobile  driving.  Individual  must  have  no  record  of  careless, 
reckless,  or  unsafe  driving. 

13.  Possess  excellent  military  appearance  and  bearing,  and  have  no  obvious  distracting 
physical  abnormalities  or  mannerisms.  Must  not  have  any  lewd  or  offensive  indelible 
marks  or  figures  (tattoos)  visible  upon  the  exposed  arm  while  wearing  the  prescribed 
duty  uniform,  to  include  the  physical  training  uniform. 
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14.  If  married  to  another  soldier,  have  a  spouse  who  will  concurrently  apply  and  be  qualified 
for  assignment  with  USAREC. 

15.  Not  currently  nor  have  been  previously  enrolled  in  the  past  12  months  in  a  drug  or  al 
dependency  intervention  program  of  any  type.  (No  waiver  is  authorized). 

16.  Not  be  pregnant  at  time  of  selection  or  prior  to  attendance  at  the  ARC. 

17.  Have  completed  the  period  of  stabilization  in  the  current  assignment. 

18.  Have  favorable  civilian  and  military  disciplinary  records.  Have  no  unfavorable  alcohol 
related  incidents  within  the  past  5  years  upon  attendance  at  the  ARC.  Examples  of  such 
disqualifying  conduct  are  driving  under  the  influence  (DUI),  driving  while  intoxicated 
(DWI),  or  charged  with  drunk  and  disorderly  conduct. 

19.  Never  have  been  convicted  by  civilian  court  or  military  court-martial. 

20.  Never  have  had  action  taken  (including  proceedings  under  the  provisions  of  Article  15, 
Uniform  Code  of  Military  Justice  (UCMJ)  by  any  authority  for:  (a)  An  offense  which 
the  maximum  penalty  under  the  UCMJ  is  death  or  confinement  for  6  months  or  more 
(No  waiver  authorized);  (b)  Any  offense  that  involves  moral  turpitude  regardless  of 
sentence  received.  (No  wavier  authorized). 

21.  Be  in  receipt  of  EDAS  assignment  instruction  to  USAREC  with  TDY  enroute  to  the 
Army  Recruiting  Course  constitutes  authority  for  ehgible  personnel  to  extend  or  reen¬ 
list  under  AR  601-280,  paragraph  3-1.  Approval  to  delete  or  defer  a  soldier  from  these 
AI  is  reserved  or  the  CDR,  PERSCOM,  ATTN:  TAPC-EPM-A. 

22.  Have  no  marital,  emotional,  or  major  medical  problems  (to  include  immediate  fam¬ 
ily)  that  would  hamper  performance  on  recruiting  duty.  Recruiting  duty  involves 
assignment  to  geographic  areas  that  are  away  from  military  medical  facilities.  Sol¬ 
diers  enrolled  in  the  Exceptional  Family  Member  Program  may  serve  as  a  recruiter. 
Every  effort  will  be  made  to  assign  them  near  a  military  installation  or  in  a  civilian 
community  where  definitive  medical  care  for  their  family  member  is  available. 

23.  Not  be  a  sole  parent  (No  wavier). 

24.  Not  have  more  than  two  dependents  (to  include  spouse)  if  a  SGT,  three  dependents 
(to  include  spouse)  if  a  SGT(P),  four  dependents  (  to  include  spouse)  if  a  SSG,  or  five 
dependents  (to  include  spouse)  if  a  SFC. 

25.  Be  financially  stable.  Have  not  filed  a  petition  claiming  bankruptcy  within  5  years  and 
not  currently  responsible  for  making  any  payments  as  a  result  of  any  such  action.  The 
financial  situation  of  soldiers  being  considered  for  selection  will  be  closely  scrutinized 
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for  soldiers  who  submit  a  DA  Form  5425-R  (Applicant/Nominee  Personal  Financial 
Statement). 

26.  If  a  volunteer  is  serving  a  dependent  restricted  tour,  the  soldier  must  waive  his  enti¬ 
tlement  to  the  home  base/advance  assignment  program. 

27.  Have  a  minimum  TIS  remaining  of  3  years  following  the  completion  of  the  Army 
Recruiting  Course. 
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